Principal components through the singular values

Topic: Principal component analysis · more questions on this topic
‹ The effect of correlation How many components ›

The centred data matrix

\mathbf{X}=\begin{pmatrix}2&1\\-2&-1\\1&-1\\-1&1\end{pmatrix}

has

\mathbf{X}^\top\mathbf{X}=\begin{pmatrix}10&2\\2&4\end{pmatrix}.

Find the eigenvalues of \mathbf{X}^\top\mathbf{X}, the singular values e_1,e_2 of \mathbf{X}, and the eigenvalues \lambda_i=e_i^2/(n-1) of the sample covariance matrix. Check that \sum_i\lambda_i=\operatorname{tr}(S).

Worked answer (click to show)

The squared singular values belong to the cross-product matrix. Divide by the covariance divisor before reporting principal-component variances.

The eigenvalues of \bigl(\begin{smallmatrix}10&2\\2&4\end{smallmatrix}\bigr) solve \lambda^2-14\lambda+36=0, so

\lambda=7\pm\sqrt{13}=10.606,\ 3.394.

The singular values of \mathbf{X} are e_1=\sqrt{10.606}=3.257 and e_2=\sqrt{3.394}=1.842. With n=4, the sample-covariance eigenvalues are

e_i^2/3=3.535\quad\text{and}\quad 1.131.

Their sum is 4.667. The sample covariance is

S=\frac13\begin{pmatrix}10&2\\2&4\end{pmatrix},

with trace (10+4)/3=4.667.