You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This isn't really an issue per se, but I found that if you wrap the entire call grad_cache.GradCache(...) in torch autocast, you will run into weird errors. This happens by default in the huggingface trainer, which wraps the call to self.compute_loss() in autocast context manager within training_step by default.
Maybe you can add a note about this gotcha in the readme?
The text was updated successfully, but these errors were encountered:
This isn't really an issue per se, but I found that if you wrap the entire call
grad_cache.GradCache(...)
in torch autocast, you will run into weird errors. This happens by default in the huggingface trainer, which wraps the call toself.compute_loss()
in autocast context manager withintraining_step
by default.Maybe you can add a note about this gotcha in the readme?
The text was updated successfully, but these errors were encountered: