Recommended way to represent Gaussian Mixture Model with gluon.probability?

[feevos]

Dear all,

the title says it: any recommended way to implement a Gaussian Mixture Model (with trainable pi, mu, log_sigma), from the tools we have in the gluon.probability.distribution?

(kindly ask help from the expert @xidulu )

Regards
Foivos

[xidulu]

@feevos
My recommended way is to use EM,
at E step, you update \pi with probability computed by MultivariateNormal
at M step, you update pi and log_sigma by running a few optimization steps on the negative log likelihood, which can also be implemented easily using distribution.MultivariateNormal.

[feevos]

Thank you @xidulu, I was after something like this from pytorch (is there something similar in gluon?):

import torch.distribution as td

def gmm_sample(pi, mus, sigmas):
        mix = td.Categorical(probs=pi)
        comp = td.Independent(td.Normal(mus, sigmas), 1)
        gmm = MixtureSameFamily(mix, comp)
                
        zz = gmm.sample()
        return zz

Again, many thanks for your help and your contributions!!

[xidulu]

Gluon.distribution currently does not have MixtureSameFamily,
however, you could perform sampling from GMM using indexing,
that is, you first sample index using OneHotCategorical and gaussian samples
then you use samples form OneHotCategorical as a mask to select a specific component:
e.g.:

>>> mask
array([[1, 0, 0],
       [0, 1, 0]])
>>> samples
array([[[-0.12913388,  1.02458042],
        [-0.04577961, -0.30007649],
        [ 0.7896362 , -0.06509621]],

       [[ 1.52198233, -0.54872673],
        [-2.45790705,  0.00642685],
        [ 0.88959479, -1.33178779]]])
>>> (mask[..., None] * samples).sum(1)
array([[-0.12913388,  1.02458042],
       [-2.45790705,  0.00642685]])

In the example above, two samples of R^2 is generated from a three component gaussian mixture.

[feevos]

Thank you - that's what I was after :)