Level sets are ellipsoids
Section of Covariance and the multivariate normal, part of Statistical learning
Level sets are ellipsoids
| Type | Section |
| Taught in | Covariance and the multivariate normal |
| Part | Statistical learning |
An ellipse drawn through a data cloud describes the spread of observations. A confidence region for the mean describes uncertainty about the centre. The second region contracts as more independent observations are collected. The population cloud does not. The contours below hold \boldsymbol{\mu} and \Sigma fixed.
Fix c>0 and consider \{\mathbf{x}:f(\mathbf{x})=c\}. If c exceeds the maximum of f, the set is empty. Otherwise divide f(\mathbf{x})=c by the leading constant and take logarithms:
-\frac12(\mathbf{x}-\boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{x}-\boldsymbol{\mu}) =\log\bigl(c\,(2\pi)^{p/2}|\Sigma|^{1/2}\bigr),
which is
(\mathbf{x}-\boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{x}-\boldsymbol{\mu})=C
for a constant C determined by c. For C>0 this is an ellipsoid centred at \boldsymbol{\mu}. In the bivariate case the level sets are ellipses. Correlation rotates and stretches them: the contour for \rho=0 is axis-aligned, the contour for \rho=0.75 leans up, and the contour for \rho=-0.75 leans down.
To find the axes, use \Sigma=\Gamma\Lambda\Gamma^\top with orthonormal columns \boldsymbol{\gamma}_1,\ldots,\boldsymbol{\gamma}_p and \Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p). Then \Sigma^{-1}=\Gamma\Lambda^{-1}\Gamma^\top. In the rotated coordinates \mathbf{u}=\Gamma^\top(\mathbf{x}-\boldsymbol{\mu}),
(\mathbf{x}-\boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{x}-\boldsymbol{\mu})=\mathbf{u}^\top\Lambda^{-1}\mathbf{u}=\sum_{j=1}^p\frac{u_j^2}{\lambda_j}=C.
This is an ellipsoid whose jth semi-axis points along \boldsymbol{\gamma}_j and has length \sqrt{C\lambda_j}. Those directions are the principal component directions derived on Principal component analysis.
Definition 1 (Squared Mahalanobis distance) The squared Mahalanobis distance of \mathbf{X} from its mean is
(\mathbf{X}-\boldsymbol{\mu})^\top\Sigma^{-1}(\mathbf{X}-\boldsymbol{\mu}).
If \mathbf{X}\sim\operatorname{N}_p(\boldsymbol{\mu},\Sigma), set \mathbf{Z}=\Sigma^{-1/2}(\mathbf{X}-\boldsymbol{\mu}), where \Sigma^{-1/2}=\Gamma\Lambda^{-1/2}\Gamma^\top is the symmetric inverse square root. The linear-image formula for covariances gives
\operatorname{Var}(\mathbf{Z})=\Sigma^{-1/2}\Sigma\Sigma^{-1/2}=I.
The next proposition says \mathbf{Z} is normal, so its coordinates are independent standard normals. The squared Mahalanobis distance is therefore
\mathbf{Z}^\top\mathbf{Z}=\sum_{j=1}^p Z_j^2\sim\chi^2_p,
the chi-squared distribution with p degrees of freedom. A density contour with C=\chi^2_{p;\alpha}, the upper \alpha point of that law, encloses probability 1-\alpha. The same upper-tail convention, with the upper probability in the first subscript, is used for the t and F critical values on Estimation and inference for the multivariate normal.
Used in
See also
Joint distribution and density · Mean vector and covariance matrix · Correlation · Multivariate normal distribution · Squared Mahalanobis distance · Linear images of a normal vector · Conditional distribution of a multivariate normal · Partial correlation