Conditional normal prediction

Topic: Covariance and the multivariate normal · more questions on this topic
‹ A Schur-complement check

For a jointly normal vector (X_1,X_2)^\top, the mean is (2,-1)^\top and the covariance matrix is

\Sigma=\begin{pmatrix}9&3\\3&4\end{pmatrix}.

You may use

\mathbf{X}_A\mid(\mathbf{X}_B=\mathbf{x}_B) \sim\operatorname{N}\bigl(\boldsymbol{\mu}_A+\Sigma_{AB}\Sigma_{BB}^{-1}(\mathbf{x}_B-\boldsymbol{\mu}_B),\; \Sigma_{AA}-\Sigma_{AB}\Sigma_{BB}^{-1}\Sigma_{BA}\bigr).

1. Determine the conditional distribution of X_1 given X_2=3. Show the substitution, not only the final parameters.

2. Let R=X_1-\mathbb{E}(X_1\mid X_2). State its distribution and explain whether it is independent of X_2.

3. Would the same independence conclusion follow from the covariance matrix alone, without joint normality? Explain.

Worked answer (click to show)

Use the Gaussian block formula for the conditional distribution, then define the residual by subtracting that conditional mean. Independence requires one more argument than a zero covariance.

1. The blocks are \mu_A=2, \mu_B=-1, \Sigma_{AA}=9, \Sigma_{AB}=3 and \Sigma_{BB}=4. The conditional mean is

2+(3/4)(3-(-1))=2+(3/4)(4)=5,

and the conditional variance is

9-3\cdot(1/4)\cdot 3=9-9/4=27/4.

Thus X_1\mid(X_2=3)\sim\operatorname{N}(5,\ 27/4).

2. The residual is the linear transformation

R=X_1-2-\tfrac34(X_2+1)

of a jointly normal vector, so R\sim\operatorname{N}(0,\ 27/4). Moreover

\operatorname{Cov}(R,X_2)=3-\tfrac34\cdot 4=0.

Joint normality turns this zero covariance into independence.

3. With only means and covariances, the analogous linear residual is uncorrelated with X_2. Conditional expectations need not be linear, and uncorrelated variables need not be independent.