🚀 Supporting deepspeed>=0.16.4's rename

[jamesbraza]

Fixes #2962

[yeruoforever]

TypeError: '>=' not supported between instances of 'list' and 'tuple'

[qgallouedec]

Thanks a lot @jamesbraza

[qgallouedec]

by the way, is this change needed as well? #2871

[jamesbraza]

by the way, is this change needed as well? #2871

Yes, testing it now. Clearly I didn't test this PR previously, as @yeruoforever reported it had a TypeError haha, my bad.

[jamesbraza]

Hi @qgallouedec I have completed my validations, this PR is ready for merge. I had to also pull in:

• 🫔 [GRPO] Pass wrapped model to unwrap_model_for_generation for DeepSpeed Stage-3 compatibility #2871 (thanks for sharing it)
• I hit [GRPO] Gradient mode issue when num_iterations > 1 #2953, and to fix it, I changed this line to with torch.no_grad() instead of with torch.inference_mode()

[qgallouedec]

Thanks, but I can't see the above mentionned changes. Did you pushed them?

[jamesbraza]

Thanks, but I can't see the above mentionned changes. Did you pushed them?

The other ones I mentioned weren't related to this PR, so they're not here. They are all here: https://github.com/Future-House/trl/tree/working-grpo-2025-02-27

[qgallouedec]

Thanks! Just commit the suggestions and we are good to merge.
Usually allowing maintainer to edit the PR makes things easier for us.

[jamesbraza]

Usually allowing maintainer to edit the PR makes things easier for us.

Yeah I always do, but for some reason that checkbox isn't present in this PR's right panel :/

Regardless, PR is ready for review again

[qgallouedec]

Thanks!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.