Identifiability
Topic of Mixtures and the EM algorithm, part of Statistical learning · Also in Analysis of variance examples
Identifiability
| Type | Topic |
| Taught in | Mixtures and the EM algorithm · Analysis of variance examples |
| Part | Statistical learning · Data analysis |
| Questions using it | 4 |
A parameter is identifiable when different values give different distributions of the data. A mixture’s labels, the effects of every level of a factor alongside an intercept, and the hidden units of a network are unidentified until a constraint fixes them.
In Mixtures and the EM algorithm
Label switching. Renaming components changes the parameter vector and leaves the density unchanged. Finite Gaussian mixtures with distinct components and positive weights are identifiable up to this permutation. Coincident components or zero weights introduce further non-identifiability (McLachlan and Peel, 2000). Before comparing component parameters across fits, decide how to match labels. Ordering means can help in one dimension. The order becomes unstable when means are close.
In Analysis of variance examples
Here \mu is a baseline mean, \alpha_i and \beta_j are the main effects of the two factors, and \gamma_{ij} is their interaction. The parameters are not all identifiable as written; R fixes them by setting the effects of the first level of each factor to zero.
Used in
- Taught in Mixtures and the EM algorithm · Analysis of variance examples
- Also used in Analysis of variance examples · Linear model examples · Model selection for stochastic models · Backpropagation · High-dimensional analysis
- Questions that use it (4) — Complexity and early stopping (answer) · Q47 (answer) · Q54 (answer) · C26 (answer)
- All 7 questions on Mixtures and the EM algorithm
See also
Finite mixture model · Hidden layer · Posterior membership · Complete-data likelihood · EM for a normal mixture · Monotonicity of EM · Likelihood of a normal mixture · What the stationary equations suggest