Do three clusters recover three species?

Section of K-means clustering, part of Statistical learning

Type	Section
Taught in	K-means clustering
Part	Statistical learning

The iris species labels are hidden while fitting. Standardisation chooses a distance measured in standard deviations. Setting K=3 uses the known number of species. The question is whether three geometric groups resemble those species, rather than whether the algorithm discovers the number three. The cross-tabulation answers that question without pretending that cluster numbers are species names. The default algorithm is used, with 50 random starts, and the objective is checked by recomputing the sum of squared distances to the reported centres.

set.seed(4006)
z <- scale(iris[, 1:4])
km <- kmeans(z, centers = 3, nstart = 50)
table(Cluster = km$cluster, Species = iris$Species)
c(within_SSE = km$tot.withinss,
  adjusted_Rand_index = mclust::adjustedRandIndex(km$cluster, iris$Species))
direct_SSE <- sum((z - km$centers[km$cluster, ])^2)
stopifnot(abs(direct_SSE - km$tot.withinss) < 1e-8)

The cross-tabulation is

Cluster   setosa   versicolor   virginica
      1        0           11          36
      2       50            0           0
      3        0           39          14

and the two summary numbers are

within_SSE   adjusted_Rand_index
  138.8884                0.6201

Setosa is recovered as its own cluster: all 50 setosa flowers sit in cluster 2, and nothing else does. Versicolor and virginica are mixed, 39 against 14 in one cluster and 11 against 36 in the other. The adjusted Rand index compares the clustering with the species partition without requiring cluster 1 to mean species 1. On its chance-corrected scale, identical partitions score 1, and the index can be negative. The value 0.6201 is the agreement on this run. The direct sum of squared distances to the assigned centres reproduces tot.withinss to the tolerance in the check, which is the unweighted criterion \widetilde W.

K-means favours compact groups in the standardised Euclidean metric. A species can have a shape, or an overlap, that this geometry does not represent. The Voronoi picture for three fixed centres in the plane is a partition into three convex cells, with boundaries that are pieces of the perpendicular bisectors. Those cells are what the assignment step builds, in the standardised four-dimensional measurement space as well as in a two-dimensional illustration.

Key idea

K-means turns a geometric preference, that nearby points share a centre, into the unweighted within-cluster sum of squares \widetilde W. Lloyd’s two steps, nearest-centre assignment and mean updates, do not increase that objective, and with the tie convention they stop at a fixed point of the alternation. The fixed point need not be the global minimum. The metric, the number of centres and the starting values each affect the partition. Check all three before interpreting a cluster label as a population.

Used in

Taught in K-means clustering
All 8 questions on K-means clustering

Used in

See also