Reconstruction from principal components
Section of Principal component analysis, part of Statistical learning
Reconstruction from principal components
| Type | Section |
| Taught in | Principal component analysis |
| Part | Statistical learning |
The sum of all component variances equals \operatorname{tr}(S), which does not depend on the basis. Maximising the variance kept by the first k components is therefore the same problem as minimising the variance left out. Let \Gamma_k=(\boldsymbol{\gamma}_1,\ldots,\boldsymbol{\gamma}_k). Replacing each centred observation by its projection \Gamma_k\Gamma_k^\top(\mathbf{x}_i-\bar{\mathbf{x}}) leaves residuals whose total squared length, divided by n-1, equals
\sum_{j=k+1}^p\lambda_j.
That value is the smallest residual attainable by any rank-k linear approximation. The leading components are at once the highest-variance coordinates and the best linear summary of the cloud in squared error. A later autoencoder network generalises the second description: a hidden layer smaller than the input tries to reproduce the input after passing through a lower-dimensional representation. Factor analysis and non-negative matrix factorisation are further representations, with different constraints.
Used in
See also
Principal component directions and scores · Principal component directions · Scaling · Principal components by Lagrange multipliers · Choosing the number of principal components · How many components, on a correlation with no response · Singular value decomposition · Iris, with and without scaling