Unequal covariance matrices

Section of Estimation and inference for the multivariate normal, part of Statistical learning

Type	Section
Taught in	Estimation and inference for the multivariate normal
Part	Statistical learning

Pooling is justified by the common-covariance model. Equal group sizes do not make that assumption true, and they do not restore the exact finite-sample F distribution under arbitrary heteroscedasticity. Large-sample robustness of the pooled procedure is a more limited conclusion. With unequal covariances the problem is the multivariate Behrens–Fisher problem, even when the group sizes happen to match.

One approximation, due to Krishnamoorthy and Yu, estimates the covariance of the mean difference directly. Define \widetilde S_l=S_l/n_l and \widetilde S=\widetilde S_1+\widetilde S_2. The statistic is

T^2=\bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr)^\top \widetilde S^{-1} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr).

For independent normal samples with n_l>p and positive definite population covariances, an F approximation is

\frac{\nu-p+1}{\nu p}\,T^2\sim F_{p,\,\nu-p+1},

with denominator degrees of freedom \nu-p+1, where

\nu=\frac{p+p^2} {\displaystyle\sum_{l=1}^{2} \dfrac{\operatorname{tr}\bigl[(\widetilde S_l\widetilde S^{-1})^2\bigr] +\bigl[\operatorname{tr}(\widetilde S_l\widetilde S^{-1})\bigr]^2}{n_l-1}}.

The traces compare each sample’s share of the total covariance estimate. The divisors n_l-1 are the covariance degrees of freedom. For p=1, put v_l=s_l^2/n_l. The formula reduces to

\nu=\frac{(v_1+v_2)^2}{v_1^2/(n_1-1)+v_2^2/(n_2-1)},

which is Welch’s degrees of freedom. The reference law remains approximate. Comparisons of several mean vectors, with a common covariance, continue as a multivariate analysis of variance: the pooled within-group matrix is the error stratum, and the between-group matrix of weighted outer products of group-mean deviations is compared with it by a further Wishart ratio.

Unequal covariance matrices

Used in

See also