R2 score

[FedericoRaimondi]

Hello,

I have a doubt on R^2 calculation.
After checking the df contribution I noticed that by recalculating R^2 I get quite different results by the one calculated with plot_model_fit.
I understand that is because we calculate Bayesian R^2 using arviz.r2_score and indeed I'm able to arrive at the same number with:

y_pred = target_scaler.inverse_transform(mmm.trace['mu'])
r2, _ = az.r2_score(y_true, y_pred)
print(r2)
>>> 0.9487456

The problem is that if I recalculate R2 by using the mean of 'mu' I would expect a similar value compared to the previous one:

y_pred = target_scaler.inverse_transform(np.mean(mmm.trace['mu'], axis=0))
r2, _ = az.r2_score(y_true, y_pred)
print(r2)
>>> 0.7293488

This is quite a big difference and it seems to be always the case, no matter the model I develop.

[pabloduque0]

I dont think a very close value is always expected with the mean of the prediction. Your posterior mu will have a certain distribution that will then generate a r2 distribution when bayesian r2 is calculated. Arviz also returns the std in case that helps.

I think this one is a good explanation of it

That said you can always rely on the metrics that work for your specific case. Eg. if you need to compare it to a non bayesian model/baseline, you can always calculate as you did in the example below.