-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Paper Benchmark Routines ? #51
Comments
This is how to run the benchmarks from which the plots are generated. https://github.com/vbharadwaj-bk/OpenEquivariance/blob/main/test/benchmark_driver.py We can follow up with more details about how to generate the plots, but first let’s see that the benchmarks will run. |
We have started providing a CLI interface to run benchmarks, which will be rolled out in a sequence of PRs (issue will be closed when all tests are runnable). These two commands will run the benchmarks as we used for our paper:
For finer control and information about more command line arguments, append For the A550 and the uvu benchmark, you will likely need to decrease the batch size from 50k to something manageable, or try running the benchmarks without the e3nn implementation which consumes more memory. Both options available from CLI. |
@mitkotak Hopefully #65 resolves this issue; our benchmark scripts are tailored for the high main memory / shared memory of the A100, but we do have some options to get plots on a GPU with lower memory. Here's a set of invocations that I tested on an A5000 (see the README.md for more details on the packages you need to run this): python benchmark.py -o outputs/uvu uvu --limited-memory --plot
python benchmark.py -o outputs/uvw uvw -b 25000 --plot
python benchmark.py -o outputs/roofline roofline --plot
python benchmark.py -o outputs/conv conv --data data/molecular_structures --limited-memory Let us know if you can run these / what issues you face, and we can get this closed. |
Have you seen this build error ?
|
Ah is this an earlier version of the package? We switched away from CMakeLists entirely; you shouldn't need to manually build the extension anymore, a fresh pip install should work. But that is a good point we should probably update or deprecate the devbuild instructions. |
I know you guys are planning to do this but just wanted to make note of it. Would appreciate if there's some set of scripts that I can trigger overnight to get the paper plots on my RTX A5500. Thanks !
The text was updated successfully, but these errors were encountered: