Rank
Let n=4 observations lie in p=6 dimensions. What is the maximum rank of the sample covariance matrix S, and how many of its eigenvalues are necessarily zero? How many principal components with positive variance can there be? If the data are instead n=4 in p=3 dimensions, what is the maximum rank?
Worked answer (click to show)
\operatorname{rank}(S)\le\min(n-1,p)=\min(3,6)=3. Since S is 6\times 6, at least 6-3=3 eigenvalues are zero. At most 3 principal components have positive variance. For n=4 and p=3, \min(n-1,p)=\min(3,3)=3, so S can have full rank 3 if the centred observations are in general position, and is then invertible.