Where the n-1 comes from

Topic: Estimation and inference for the multivariate normal · more questions on this topic
Pooling two groups ›

Show that

\sum_{i=1}^n(\mathbf{X}_i-\boldsymbol{\mu})(\mathbf{X}_i-\boldsymbol{\mu})^\top =A+n(\overline{\mathbf{X}}-\boldsymbol{\mu})(\overline{\mathbf{X}}-\boldsymbol{\mu})^\top,

and use it to prove \mathbb{E}(A)=(n-1)\Sigma. Which assumption about the \mathbf{X}_i does the proof use beyond a common mean and covariance?

Worked answer (click to show)

The sample mean is random, so centring at it changes the expected scatter. Isolate that change before taking expectations.

Write \mathbf{X}_i-\boldsymbol{\mu}=(\mathbf{X}_i-\overline{\mathbf{X}})+(\overline{\mathbf{X}}-\boldsymbol{\mu}) and expand the outer product. The cross terms are

\sum_i(\mathbf{X}_i-\overline{\mathbf{X}})(\overline{\mathbf{X}}-\boldsymbol{\mu})^\top=\mathbf{0}\,(\overline{\mathbf{X}}-\boldsymbol{\mu})^\top=\mathbf{0}

and its transpose, so

\sum_i(\mathbf{X}_i-\boldsymbol{\mu})(\mathbf{X}_i-\boldsymbol{\mu})^\top =A+n(\overline{\mathbf{X}}-\boldsymbol{\mu})(\overline{\mathbf{X}}-\boldsymbol{\mu})^\top.

Take expectations. The left side has expectation n\Sigma. For the last term,

\mathbb{E}\bigl[(\overline{\mathbf{X}}-\boldsymbol{\mu})(\overline{\mathbf{X}}-\boldsymbol{\mu})^\top\bigr] =\operatorname{Var}(\overline{\mathbf{X}})=\Sigma/n,

which uses uncorrelatedness of the \mathbf{X}_i:

\operatorname{Var}(\overline{\mathbf{X}})=n^{-2}\sum_{i,l}\operatorname{Cov}(\mathbf{X}_i,\mathbf{X}_l)=n^{-2}\cdot n\Sigma=\Sigma/n.

Hence n\Sigma=\mathbb{E}(A)+n\cdot(\Sigma/n), that is \mathbb{E}(A)=(n-1)\Sigma. Normality is not used. The proof uses a common mean, a common covariance, and uncorrelated observations.