The effect of correlation

Topic: Principal component analysis · more questions on this topic
‹ Properties of the components Principal components through the singular values ›

For p=2 and correlation matrix

R=\begin{pmatrix}1&r\\r&1\end{pmatrix}

with r>0, show that the eigenvalues are 1\pm r for every r, and that the eigenvectors do not depend on r. What happens to the proportion of variance explained by the first component as r\to 0 and as r\to 1?

Worked answer (click to show)

For positive correlation, the sum direction collects shared variation. The limiting cases say what happens when that shared variation disappears or becomes perfect.

|R-\lambda I|=(1-\lambda)^2-r^2=0

gives \lambda=1\pm r. Since

R(1,1)^\top=(1+r)(1,1)^\top, \qquad R(1,-1)^\top=(1-r)(1,-1)^\top,

the eigenvectors are (1,1)^\top/\sqrt{2} and (1,-1)^\top/\sqrt{2} for every r>0. The proportion explained by the first component is (1+r)/2. As r\to 0 it tends to 1/2: with R\to I no direction is preferred, every direction is an eigenvector, and the first component is determined only by how ties are broken. As r\to 1 it tends to 1: the two standardised variables become the same variable and one component carries all the variance.