Iris: one species or three

Section of Covariance and the multivariate normal, part of Statistical learning

Type	Section
Taught in	Covariance and the multivariate normal
Part	Statistical learning

The four iris measurements share centimetre units. Pooling the species changes the question. The calculation below compares the covariance of all 150 flowers with the covariance within setosa, and then checks that the sample conditional-mean slope matches the least-squares slope. A single Gaussian approximation to the pooled data ignores the species structure. In the code, iris_x holds the four measurements and iris_y holds the species labels.

iris_x <- iris[, 1:4]
iris_y <- iris$Species
round(cov(iris_x), 3)
round(cov(subset(iris, Species == "setosa")[, 1:4]), 3)
aggregate(. ~ Species, data = iris, FUN = mean)

setosa <- subset(iris, Species == "setosa")
x <- setosa$Sepal.Length
y <- setosa$Sepal.Width
ols <- coef(lm(y ~ x))
S <- cov(cbind(x, y))
slope <- S[2, 1] / S[1, 1]
gauss <- c(intercept = mean(y) - slope * mean(x), slope = slope)
rbind(ols = unname(ols), gaussian = unname(gauss))
stopifnot(max(abs(unname(ols) - unname(gauss))) < 1e-10)

pairs(iris[, 1:4], col = as.integer(iris_y), pch = 19, cex = 0.55)

The pooled sample covariance, rounded to three decimals, is

             Sepal.Length Sepal.Width Petal.Length Petal.Width
Sepal.Length        0.686      -0.042        1.274       0.516
Sepal.Width        -0.042       0.190       -0.330      -0.122
Petal.Length        1.274      -0.330        3.116       1.296
Petal.Width         0.516      -0.122        1.296       0.581

Within setosa the sample covariance is much smaller, and the sepal and petal blocks are only weakly associated:

             Sepal.Length Sepal.Width Petal.Length Petal.Width
Sepal.Length        0.124       0.099        0.016       0.010
Sepal.Width         0.099       0.144        0.012       0.009
Petal.Length        0.016       0.012        0.030       0.006
Petal.Width         0.010       0.009        0.006       0.011

The species means, in centimetres, are

     Species Sepal.Length Sepal.Width Petal.Length Petal.Width
      setosa        5.006       3.428        1.462       0.246
  versicolor        5.936       2.770        4.260       1.326
   virginica        6.588       2.974        5.552       2.026

On setosa, sepal width against sepal length, the least-squares fit and the slope taken from the sample covariance agree:

             intercept   slope
ols           -0.5694  0.7985
gaussian      -0.5694  0.7985

The scatterplot matrix of the four measurements, coloured by species, shows three clouds. The colours are the observed labels. No model has been fitted to produce them.

The pooled covariance includes both within-species variation and variation between species means. The law of total covariance makes the split explicit. If G denotes the species,

\operatorname{Cov}(\mathbf{X})=\mathbb{E}\bigl\{\operatorname{Cov}(\mathbf{X}\mid G)\bigr\}+\operatorname{Cov}\bigl\{\mathbb{E}(\mathbf{X}\mid G)\bigr\}.

A strong pooled correlation can therefore be a difference between group means. The setosa block shows what remains when that difference is removed.

Key idea

Conditioning a multivariate normal vector on some of its coordinates gives another multivariate normal. The conditional mean is affine in the conditioning values, and the conditional covariance is the Schur complement \Sigma_{AA}-\Sigma_{AB}\Sigma_{BB}^{-1}\Sigma_{AB}^\top, which does not depend on the observed values. Zero covariance is independence inside this family. The iris calculation shows why the population has to be named first: the pooled covariance mixes within-species scatter with the gaps between species means.

Iris: one species or three

Used in

See also