Iris, with and without scaling

Section of Principal component analysis, part of Statistical learning

Type	Section
Taught in	Principal component analysis
Part	Statistical learning

The fit uses every iris flower, and it is descriptive. Species labels colour the plot after the fit. They do not enter S. Correlation principal components standardise each measurement to unit variance before the eigendecomposition. Covariance principal components keep the original variances, so a measurement with larger sample variance takes a larger share of the first direction. That is the unit dependence in the scaling remark: petal length, whose sample variance in the pooled covariance is about 3.12, dominates a covariance analysis, while standardisation gives each measurement the same marginal scale.

iris_x <- iris[, 1:4]
iris_y <- iris$Species
pc_cov <- prcomp(iris_x, center = TRUE, scale. = FALSE)
pc_cor <- prcomp(iris_x, center = TRUE, scale. = TRUE)
round(rbind(Covariance = pc_cov$sdev^2 / sum(pc_cov$sdev^2),
            Correlation = pc_cor$sdev^2 / sum(pc_cor$sdev^2)), 3)
round(pc_cor$rotation[, 1:2], 3)
stopifnot(max(abs(cov(pc_cor$x) - diag(pc_cor$sdev^2))) < 1e-10)
plot(pc_cor$x[, 1:2], col = as.integer(iris_y), pch = 19,
     xlab = "PC1 score", ylab = "PC2 score")

The proportions of variance are

              PC1   PC2   PC3   PC4
Covariance  0.925 0.053 0.017 0.005
Correlation 0.730 0.229 0.037 0.005

The first two standardised components explain 95.8\% of the total standardised variance: the partial sum of the eigenvalues, divided by the trace, before the row is rounded for display. The leading two correlation loadings, rounded to three decimals, are

                PC1     PC2
Sepal.Length  0.521  -0.377
Sepal.Width  -0.269  -0.923
Petal.Length  0.580  -0.024
Petal.Width   0.565  -0.067

The sign of a loading may be flipped without changing the line it spans or the variance it carries. The check cov(pc_cor$x) against diag(pc_cor$sdev^2) confirms that the sample covariance of the scores is diagonal, with the reported component variances on the diagonal. The score plot colours each flower by species after the fit. Separation of species in that plot describes this sample. A prediction experiment has to estimate the centring, the scaling, and the directions inside each training fold, and apply that transformation unchanged to the validation rows.

Key idea

The principal component directions are the eigenvectors of the sample covariance, ordered by eigenvalue. The jth eigenvalue is the sample variance of the jth component, so the proportion of variance explained by the first k components is \sum_{j\le k}\lambda_j\big/\sum_{j\le p}\lambda_j. The same directions are the right singular vectors of the centred data matrix. Scaling changes the matrix being diagonalised, and a principal axis is not a regression line: the axis maximises explanatory variance, while a regression line minimises squared error in a response.

Used in

Taught in Principal component analysis
All 7 questions on Principal component analysis

Used in

See also