Counting parameters

Topic: Mixtures and the EM algorithm · more questions on this topic
‹ An unbounded likelihood Label switching ›

Using d=Hp+Hp(p+1)/2+(H-1), find the number of free parameters for H=3 components in p=2 and p=5 dimensions. How many if the covariance matrices are constrained to be diagonal, and how many if they are all equal to one common full matrix?

Worked answer (click to show)

Count means, covariance entries, and independent mixing weights separately. Symmetry reduces a full covariance count, and the weights lose one degree of freedom because they sum to one.

For p=2 and H=3, the unrestricted count is 6+3\cdot 3+2=17. For p=5 and H=3 it is 15+3\cdot 15+2=62.

Diagonal covariances replace p(p+1)/2 by p per component: 6+6+2=14 when p=2, and 15+15+2=32 when p=5.

A common full matrix has p(p+1)/2 covariance parameters in total: 6+3+2=11 when p=2, and 15+15+2=32 when p=5.