New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to get the total training step time？ #63

Open

leondada opened this issue Jan 7, 2025 · 1 comment

leondada commented Jan 7, 2025

it seems that the workload generated by AICB describes the behavior of a single GPU.

How can we use this workload to simulate the total training time for a complete step? I cannot find the corresponding code for this logic.

Collaborator

Huoyuan100861 commented Jan 10, 2025

it seems that the workload generated by AICB describes the behavior of a single GPU.

How can we use this workload to simulate the total training time for a complete step? I cannot find the corresponding code for this logic.

Actually, in a normal training process, each GPU completes an iteration simultaneously (they synchronize at the end).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment