Uncertainty about mixture parameters
Section of Mixtures and the EM algorithm, part of Statistical learning
Uncertainty about mixture parameters
| Type | Section |
| Taught in | Mixtures and the EM algorithm |
| Part | Statistical learning |
Standard errors can be built from the observed information I(\widehat\Psi;X_{\mathrm{all}}), expressed through the complete-data information and the complete and incomplete scores (McLachlan and Peel, 2000). The algebra is heavy, and it still replaces the expected information by an observed-information approximation.
A Bayesian analysis assigns a distribution to the unknown parameters and updates it with the likelihood. Its influence depends on the prior, the amount of information in the data, and the model geometry. There is no general rule that a prior equals one extra observation. Mixtures require particular care around weakly identified components. Integration against a prior is also the starting point for the model-comparison criterion on the model-selection page.
The parametric bootstrap asks what would happen if the fitted mixture generated a new sample of the same size. Refit to each simulated sample and examine the variation in the estimates. This conditions on the chosen family and on H. Resimulating a mistaken component model does not detect the mistake. Near singular fits, ordinary bootstrap uncertainty summaries also need care.
Draw B bootstrap samples X^{(1)},\ldots,X^{(B)}, each of size n, from the fitted mixture. For each observation, draw a component label with probabilities \widehat\pi_h, then draw the observation from that component’s normal distribution. Fit EM to each bootstrap sample. The fits are B parameter vectors \widehat\Psi^{(1)},\ldots,\widehat\Psi^{(B)}.
The bootstrap covariance is the sample covariance of those vectors,
\widehat{\mathrm{Cov}}_B(\widehat\Psi) =\frac{1}{B-1}\sum_{b=1}^{B} \bigl(\widehat\Psi^{(b)}-\overline\Psi\bigr) \bigl(\widehat\Psi^{(b)}-\overline\Psi\bigr)^\top, \qquad \overline\Psi=\frac1B\sum_{b=1}^{B}\widehat\Psi^{(b)}.
The labels are defined only up to the permutation above. Before the bootstrap vectors enter the covariance, match the components of each bootstrap fit to the components of \widehat\Psi, for example by closeness of the means. Without that match, label switching inflates the covariance: the average mixes different components.
Used in
See also
Finite mixture model · Posterior membership · Complete-data likelihood · EM for a normal mixture · Monotonicity of EM · Likelihood of a normal mixture · What the stationary equations suggest · E and M steps