Whitening and its limits

Topic: Estimation and inference for the multivariate normal · more questions on this topic
‹ A two-sample statistic Normal conditioning and whitening ›

A dataset has n=80 observations in p=4 dimensions, sample mean \bar{\mathbf{x}} and positive definite sample covariance S=U\Lambda U^\top. Define \mathbf{z}_i=S^{-1/2}(\mathbf{x}_i-\bar{\mathbf{x}}).

  1. Explain how S^{-1/2} is obtained from the spectral decomposition.

  2. Determine the sample mean and covariance of the \mathbf{z}_i, showing the matrix calculation.

  3. Explain why this operation need not produce independent normally distributed coordinates, and what fails if p=100 with the same n.

Whitening is an exact statement about the covariance used in the transformation. Work through the matrix identity before deciding what it says about a population distribution.

  1. From S=U\Lambda U^\top with positive eigenvalues and an orthonormal eigenbasis,

S^{-1/2}=U\operatorname{diag}(\lambda_1^{-1/2},\ldots,\lambda_4^{-1/2})U^\top.

  1. The transformed sample mean is \bar{\mathbf{z}}=S^{-1/2}(\bar{\mathbf{x}}-\bar{\mathbf{x}})=\mathbf{0}. With the same n-1 covariance divisor,

S_z=\frac{1}{n-1}\sum_i S^{-1/2}(\mathbf{x}_i-\bar{\mathbf{x}})(\mathbf{x}_i-\bar{\mathbf{x}})^\top S^{-1/2} =S^{-1/2}SS^{-1/2}=I_4.

  1. These are empirical first- and second-moment statements. A linear transform does not generally make a non-normal distribution normal, and zero sample correlations do not establish population independence. When p=100 and n=80, \operatorname{rank}(S)\le 79, so an ordinary inverse square root in all 100 dimensions does not exist. Reduction to the positive-eigenvalue subspace, or a regularisation, changes the construction.