Pooling two groups

Topic: Estimation and inference for the multivariate normal · more questions on this topic
‹ Where the n-1 comes from Two factors of one covariance ›

Group 1 has n_1=10 observations with mean (3,2)^\top and

A_1=\begin{pmatrix}18&6\\6&12\end{pmatrix}.

Group 2 has n_2=15 observations with mean (1,1)^\top and

A_2=\begin{pmatrix}30&-9\\-9&24\end{pmatrix}.

Compute the pooled covariance matrix with divisor n-G. Then compute the sample covariance matrix of the combined sample of 25 observations about its grand mean, using

A_{\mathrm{total}}=A_1+A_2+\frac{n_1 n_2}{n}(\bar{\mathbf{x}}_1-\bar{\mathbf{x}}_2)(\bar{\mathbf{x}}_1-\bar{\mathbf{x}}_2)^\top

and divisor n-1, and explain why the two differ.

Worked answer (click to show)

Pooling within groups and ignoring the group labels answer different questions. The common covariance is meant to describe the within-group scatter.

With n=25, G=2 and n-G=23,

S=\frac{1}{23}\left( \begin{pmatrix}18&6\\6&12\end{pmatrix} +\begin{pmatrix}30&-9\\-9&24\end{pmatrix} \right) =\frac{1}{23}\begin{pmatrix}48&-3\\-3&36\end{pmatrix} =\begin{pmatrix}2.087&-0.130\\-0.130&1.565\end{pmatrix}.

For the combined sample, \bar{\mathbf{x}}_1-\bar{\mathbf{x}}_2=(2,1)^\top and n_1 n_2/n=150/25=6, so

A_{\mathrm{total}} =\begin{pmatrix}48&-3\\-3&36\end{pmatrix} +6\begin{pmatrix}4&2\\2&1\end{pmatrix} =\begin{pmatrix}48&-3\\-3&36\end{pmatrix} +\begin{pmatrix}24&12\\12&6\end{pmatrix} =\begin{pmatrix}72&9\\9&42\end{pmatrix}.

Dividing by n-1=24,

\frac{A_{\mathrm{total}}}{24}=\begin{pmatrix}3&0.375\\0.375&1.75\end{pmatrix}.

The combined-sample estimate is larger on the diagonal and has the opposite sign for the covariance. It mixes the within-group scatter, which is what \Sigma describes, with the between-group scatter caused by the different means, which \Sigma does not describe. Pooling centres each group at its own mean first.