Is it possible to set coefficient constraints for extra features?

[saukang]

Hi all,

Does anybody else attempt to set constraints for the coefficient of extra features? For instance, I have 6 extra features here, the first one is Competitor Spend data which I expect to have a negative impact on Sales/Revenue, whereas the other 5 extra features are non-marketing activities but are expected to boost the baseline sales. In short, is there a way to set the first extra feature coefficient to be strictly lesser than 0 and the coefficients for other extra features to be strictly greater than 0?

Best regards,
Sau Kang.

[pabloduque0]

Hello @saukang ! Thanks for your question!

The way of restricting the values of a parameter would be through the distribution of the pior. For example if you want only positive values you could use a HalfNormal distribution.

Unfortunately at the moment we do not allow for multiple distributions for the extra features (that be one that restricts for positive and another for negative values). You can have different values for the same distribution but not different distributions. However this is something we are likely to add in upcoming versions.

However a workaround in the meantime will be setting different values of the prior for each extra feature (with the same distribution). For example having a Normal distribution centered at -X for the first feature and the rest centered at Y. Something along the following:

x, y = # set your values here
loc_values = jnp.array([-x, y, y, y, y, y])
scale_values = jnp.array([ # set your values here as well])
custom_priors = {"extra_features": numpyro.distribution.Normal(loc=loc_values, scale=scale_values)}
mmm.fit(..., custom_priors=custom_priors)

[saukang]

Could you explain more about the x and y values that you have mentioned in your example code provided? Like what values should they be? I have tried looking up the Custom Priors page from the documentation and even NumPryo's too but still couldn't understand the meaning of loc and scale parameters, perhaps if there are any related resources that can answer my doubts?

Speaking of future features, it would be great to consider adstock transformation on some of the extra_features as well as I see many other MMM users plugging 'Competitor Spend' (just like what I did) as part of their MMM model which they expect it to have some sort of adstock effect too. Not sure if that is in your upcoming feature list or not. 😁

Anyhow, thanks for your reply and suggestion! So excited to see further development from Lightweight and I can't wait! I just started my data science journey not long ago and apologies for asking so many questions 🙏 Thanks in advance!

Best regards,
Sau Kang.

[pabloduque0]

I did not want to specify specific values since I dont know how the data looks in your case and did not want to give bad priors as an example. That said those values could look something like the following, but please know that Im just making this values up as I do not know how your data looks.

x, y = -2, 2
loc_values = jnp.array([x, y, y, y, y, y])
scale_values = jnp.ones(6) # Maybe you can keep all your sigmas to 1.
custom_priors = {"extra_features": numpyro.distribution.Normal(loc=loc_values, scale=scale_values)}
mmm.fit(..., custom_priors=custom_priors)

This wont restrict your values to positive and negative respectively but it should skew them towards such.

loc is the location or mean of the Normal distribution, and scale refers to the sigma of the Normal distribution. Take a look at any resources about such distribution, like this one.

Yes, that is a good point. We might be able to add transformations to the extra features as well, although that might take slightly longer.

[saukang]

Ahhh if loc refers to mean and scale refers to sigma then it makes a lot of sense! Thank you so much @pabloduque0, for being so helpful! Have a great day!