Linear images and marginals
Section of Covariance and the multivariate normal, part of Statistical learning
Linear images and marginals
| Type | Section |
| Taught in | Covariance and the multivariate normal |
| Part | Statistical learning |
Proposition 1 (Linear images of a normal vector) Let \mathbf{X}\sim\operatorname{N}_p(\boldsymbol{\mu},\Sigma) with \Sigma of rank p, and let A be an m\times p matrix of rank m\le p. Then \mathbf{Y}=A\mathbf{X} satisfies
\mathbf{Y}\sim\operatorname{N}_m(A\boldsymbol{\mu},\,A\Sigma A^\top).
The mean and covariance are the general transformation rules. The additional claim is that the image is normal. It is the fact used whenever a margin, a sum, or a residual of a normal vector is treated as normal.
A binary matrix of zeros and ones extracts a margin. With
A=\begin{pmatrix}1&0&0\\0&1&0\end{pmatrix},
the product A\mathbf{X} equals (X_1,X_2)^\top, and the marginal covariance is the corresponding 2\times 2 block of \Sigma. Any selection matrix, with a single 1 in each row, shows that every subset of the coordinates of a multivariate normal vector is multivariate normal, with the corresponding sub-vector of \boldsymbol{\mu} and sub-matrix of \Sigma. The matrix A=(I_q\ 0) selects the first q coordinates.
The converse fails. Normal margins do not force the joint distribution to be multivariate normal.
Example 1 (Normal margins without a normal joint) Let X\sim\operatorname{N}(0,1) and let W be independent of X with \mathbb{P}(W=1)=\mathbb{P}(W=-1)=\tfrac12. Set Y=WX.
Question. Show that Y\sim\operatorname{N}(0,1) and \operatorname{Cov}(X,Y)=0, and that (X,Y) is not bivariate normal.
Solution. For any y,
\mathbb{P}(Y\le y)=\tfrac12\mathbb{P}(X\le y)+\tfrac12\mathbb{P}(-X\le y)=\Phi(y),
because -X\sim\operatorname{N}(0,1) as well. So Y\sim\operatorname{N}(0,1). The covariance is
\mathbb{E}(XY)=\mathbb{E}(WX^2)=\mathbb{E}(W)\mathbb{E}(X^2)=0,
where the product splits because W is independent of X. Now
X+Y=(1+W)X
equals 2X with probability \tfrac12 and equals 0 with probability \tfrac12. The sum has an atom at 0 of mass \tfrac12. Every linear image of a bivariate normal vector is normal, possibly degenerate. A degenerate normal is a constant, and the only constant here would require the combination to be identically zero, which X+Y is not. An atom of mass \tfrac12 at 0, together with a spread-out half, is not a normal distribution. Therefore (X,Y) is not bivariate normal. The pair has normal margins and zero covariance, and it is dependent because |Y|=|X|.
Used in
See also
Joint distribution and density · Mean vector and covariance matrix · Correlation · Multivariate normal distribution · Squared Mahalanobis distance · Linear images of a normal vector · Conditional distribution of a multivariate normal · Partial correlation