Iris mixture membership

Section of Mixtures and the EM algorithm, part of Statistical learning

Iris mixture membership

Type Section
Taught in Mixtures and the EM algorithm
Part Statistical learning

The fit below is a three-component Gaussian mixture on the two petal measurements, with H held fixed. Fixing H separates the membership probabilities from the choice of how many components to use. Let iris_x be the 150\times 4 matrix of iris measurements in the usual order (sepal length, sepal width, petal length, petal width), and let iris_y be the species labels. Columns 3 and 4 are the petal measurements. The name VVV asks each component for its own unconstrained covariance, which is the normal mixture derived above.

set.seed(5006)
mix <- mclust::Mclust(iris_x[, 3:4], G = 3,
                      modelNames = "VVV", verbose = FALSE)
table(Component = mix$classification, Species = iris_y)
#>           Species
#> Component setosa versicolor virginica
#>         1      50          0         0
#>         2       0         49         3
#>         3       0          1        47
round(head(mix$z), 3)
#>      [,1] [,2] [,3]
#> [1,]    1     0    0
#> [2,]    1     0    0
#> [3,]    1     0    0
#> [4,]    1     0    0
#> [5,]    1     0    0
#> [6,]    1     0    0
summary(1 - apply(mix$z, 1, max))
#>    Min. 1st Qu. Median   Mean 3rd Qu.   Max.
#>  0.0000  0.0000 0.0003 0.0310  0.0175 0.4596
stopifnot(max(abs(rowSums(mix$z) - 1)) < 1e-10)

Each row of mix$z is the posterior membership at the fitted parameter. The classification table counts 50 setosa in component 1, 49 versicolor and 3 virginica in component 2, and 1 versicolor and 47 virginica in component 3. One minus the largest membership has minimum 0, median 0.0003, mean 0.0310, and maximum 0.4596. That summary is the uncertainty of the assigned component given the fitted parameter and given H=3. It leaves out uncertainty about \Psi and about H. Permuting the component labels leaves the fitted density unchanged, which is why bootstrap components have to be matched before their parameters are averaged.

Key idea

EM replaces a log-of-a-sum by the expected complete-data log-likelihood. The E step computes the soft memberships \tau_{ih}. The M step is the weighted sample proportion, mean, and covariance. An exact step cannot decrease the observed-data log-likelihood. The same unconstrained likelihood is unbounded: a component covariance can collapse onto sample points, so a rising likelihood is checked against the smallest covariance eigenvalues and is not, by itself, a global maximum.

Used in

See also

Finite mixture model · Posterior membership · Complete-data likelihood · EM for a normal mixture · Monotonicity of EM · Likelihood of a normal mixture · What the stationary equations suggest · E and M steps

Glossary and index