Understanding the differences between fit_gpytorch_mll, fit_gpytorch_mll_torch and fit_gpytorch_mll_scipy

[theo-brown]

I'm trying to fit a particular GP model.
When I use fit_gpytorch_mll, I get ModelFittingError: All attempts to fit the model have failed.
However, I'm able to fit it using fit_gpytorch_mll_torch with no problems.

The code around the fit_gpytorch_mll function is quite difficult to understand because it's so modular. I would've assumed that, if fit_gpytorch_mll calls fit_gpytorch_mll_torch, it uses the default values -- so I'd expect the same behaviour.
Running it in a with debug(True) context doesn't produce any more clues as to why fit_gpytorch_mll is failing.

Could someone explain to me why it might be working with fit_gpytorch_mll_torch and not fit_gpytorch_mll?

Thanks for your help!

Specific context: I'm developing a custom GPyTorch composition-based kernel for a particular application.
I'm testing it with a SingleTaskGP, using a Matern-52 as the base kernel for my composition kernel, with the same GammaPriors as in the default BoTorch options. These priors should work for the composition kernel I'm using with no changes.

[Balandat]

@theo-brown would you have a repro for how things are failing for you?

By default, fit_gpytorch_mll won't dispatch to fit_gpytorch_mll_torch but instead fit_gpytorch_mll_scipy. What happens if you call fit_gpytorch_mll_scipy on your problem directly?