Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks and benchmark tests #10

Open
lalalune opened this issue Jul 25, 2023 · 0 comments
Open

Add benchmarks and benchmark tests #10

lalalune opened this issue Jul 25, 2023 · 0 comments

Comments

@lalalune
Copy link
Collaborator

We have a list of benchmark tasks which we'd like to run to validate that changes are having a positive impact on the quality of output.

The benchmarks will test that the tasks completed at all and measure how many epochs they took to complete.

Benchmarks will take a while to run and could be pretty expensive so we should make sure we have controls for managing this and cutoffs for maximum epochs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant