Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite and test workflow #176

Open
lars-t-hansen opened this issue May 27, 2024 · 3 comments
Open

Test suite and test workflow #176

lars-t-hansen opened this issue May 27, 2024 · 3 comments
Labels

Comments

@lars-t-hansen
Copy link
Collaborator

I didn't want to make this part of #172 since it's broader. Currently:

  • the github workflow runs cargo build but then each test case in tests/ also does that; the latter are redundant
  • with that in mind, the building of the C test helpers in the test dir should be done once by the workflow probably
  • we may want a toplevel build.sh script that does all of that for us, for convenience when developing
  • we want test cases to test the output format of sonar ps better. right now there's some ad-hoccery around some of the fields, we should test more fields, in particular timestamp (cf Regression: Timestamp format is wrong #173, currently an internal test only)
  • (more ideas here)
@lars-t-hansen lars-t-hansen changed the title Test suite Test suite and test workflow Oct 7, 2024
@lars-t-hansen
Copy link
Collaborator Author

More ideas:

  • an acceptance criterion for release is to pass local testing on hardware of every GPU type, which means running it manually on these systems
  • ideally we would have some automation to run testing daily on hardware of every GPU type and report problems by email or other push notification

@lars-t-hansen
Copy link
Collaborator Author

The various cargo build runs are less redundant than it may seem, since some build enable various feature sets, and if we're not going to build for every test then the runs that do change the features need to be very careful to restore the original binary on all exit paths.

@lars-t-hansen
Copy link
Collaborator Author

Also see #217.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant