Normal conditioning and whitening

Topic: Estimation and inference for the multivariate normal · more questions on this topic
‹ Whitening and its limits A transformation is not a distributional assumption ›

You may use the conditional-normal formula

\mathbf{X}_A\mid(\mathbf{X}_B=\mathbf{x}_B) \sim\operatorname{N}\bigl(\boldsymbol{\mu}_A+\Sigma_{AB}\Sigma_{BB}^{-1}(\mathbf{x}_B-\boldsymbol{\mu}_B),\; \Sigma_{AA}-\Sigma_{AB}\Sigma_{BB}^{-1}\Sigma_{BA}\bigr).

  1. Let (X_1,X_2)^\top\sim\operatorname{N}_2\bigl((1,2)^\top,\ \begin{pmatrix}4&2\\2&9\end{pmatrix}\bigr). Derive the conditional distribution of X_1 given X_2=5.

  2. For this model, explain the distribution of X_1-\mathbb{E}(X_1\mid X_2) and whether changing the observed value of X_2 changes its variance.

  3. A different dataset has positive definite sample covariance S. Describe the sample mean and covariance after applying \mathbf{z}_i=S^{-1/2}(\mathbf{x}_i-\bar{\mathbf{x}}).

Identify the response block and the conditioning block first. The subtraction in the conditional variance is the reduction supplied by the observed coordinate.

  1. Here \Sigma_{AB}=2 and \Sigma_{BB}=9. The conditional mean is

1+\frac{2}{9}(5-2)=\frac53,

and the conditional variance is

4-\frac{2^2}{9}=\frac{32}{9}.

Hence X_1\mid(X_2=5)\sim\operatorname{N}(5/3,\ 32/9).

  1. The residual is a linear combination of jointly normal variables, with mean zero and variance 32/9. Its covariance with X_2 is 2-(2/9)\cdot 9=0, so it is independent of X_2. The conditional residual variance therefore does not depend on the observed x_2. This uses the specified Gaussian model. It is not a property of residuals from every fitted model.

  2. The empirical mean is zero and the empirical covariance is I_p, using the same covariance divisor before and after the transformation.