ci: use Testing Farm instead of GH Actions + MacOS + Vagrant #95
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The current solution to run a VM on MacOS shared runners using Vagrant is becoming very unreliable and almost always breaks. Replace it with Testing Farm [1], utilizing the "Schedule tests on Testing Farm" GH Action [2]. Advantages: - more reliable - allows us to test also on aarch64 - currently no usage limits for public projects (will likely change) Disadvantages: - requires an API key to be stored in the project's secrets (under the name TESTING_FARM_API_TOKEN) - GitHub won't allow the target project's secrets to be used when running a workflow on a pull request from a fork, therefore PRs won't be automatically tested unless the contributor sets their own API key in their fork - only people with a Fedora account in the fedora-contributor group can currently obtain an API key on their own (others would need to ask for it via email) - no real-time view of test progress (may become available in the future via the artifacts view) The new CI runs the testsuite on a similar testing matrix as the old one, although it only tests on the latest Fedora version and additionally tests on the aarch64 architecture. It also runs the NFS tests (./tools/nfs.sh), which the old one didn't. [1] https://testing-farm.io/ [2] https://github.com/marketplace/actions/schedule-tests-on-testing-farm Signed-off-by: Ondrej Mosnacek <[email protected]>
WOnder93
force-pushed
the
testing-farm-v2
branch
from
b178a03 to
7cf42df
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current solution to run a VM on MacOS shared runners using Vagrant is becoming very unreliable and almost always breaks. Replace it with Testing Farm [1], utilizing the "Schedule tests on Testing Farm" GH Action [2].
Advantages:
Disadvantages:
The new CI runs the testsuite on a similar testing matrix as the old one, although it only tests on the latest Fedora version and additionally tests on the aarch64 architecture. It also runs the NFS tests (./tools/nfs.sh), which the old one didn't.
[1] https://testing-farm.io/
[2] https://github.com/marketplace/actions/schedule-tests-on-testing-farm