Pooled covariance
Section of Estimation and inference for the multivariate normal, part of Statistical learning
Pooled covariance
| Type | Section |
| Taught in | Estimation and inference for the multivariate normal |
| Part | Statistical learning |
| Questions using it | 3 |
The common \Sigma is estimated from both samples. Every observation informs that covariance, and the two means are not assumed equal under the alternative, so each sample is centred at its own mean. Estimating two means removes two degrees of freedom:
S=\frac{1}{n_1+n_2-2} \Biggl( \sum_{i=1}^{n_1}(\mathbf{x}_i^{(1)}-\bar{\mathbf{x}}^{(1)})(\mathbf{x}_i^{(1)}-\bar{\mathbf{x}}^{(1)})^\top +\sum_{i=1}^{n_2}(\mathbf{x}_i^{(2)}-\bar{\mathbf{x}}^{(2)})(\mathbf{x}_i^{(2)}-\bar{\mathbf{x}}^{(2)})^\top \Biggr).
This is the pooled formula with G=2. The Wishart statement applied to the pooled residuals gives
(n_1+n_2-2)S\sim W_p(\Sigma,\,n_1+n_2-2),
independent of both sample means and therefore of their difference. The one-sample passage from a normal vector and an independent Wishart matrix to an F statistic used \sqrt{n}(\overline{\mathbf{X}}-\boldsymbol{\mu}_0) and n-1 degrees of freedom. Replace those ingredients by the mean-difference law and by n_1+n_2-2 degrees of freedom. The resulting statistic is
T^2=\frac{n_1 n_2}{n_1+n_2} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr)^\top S^{-1} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr),
and it follows the T^2 law on n_1+n_2-2 degrees of freedom. The same rescaling as in the one-sample case produces
\frac{n_1+n_2-p-1}{(n_1+n_2-2)\,p}\,T^2 \sim F_{p,\,n_1+n_2-p-1}.
Theorem 1 (Two-sample Hotelling test) Under the common-covariance normal model above, reject H_0:\boldsymbol{\mu}_1=\boldsymbol{\mu}_2 at level \alpha when
\frac{n_1+n_2-p-1}{(n_1+n_2-2)\,p}\,T^2 >F_{\alpha;\,p,\,n_1+n_2-p-1}.
The second degree of freedom is n_1+n_2-p-1. It is the Wishart degrees of freedom n_1+n_2-2, reduced by the dimension in the same way that n-1 became n-p in the one-sample F.
Used in
- Taught in Estimation and inference for the multivariate normal
- Also used in Covariance and the multivariate normal · Principal component analysis · Statistical classifiers and discriminant analysis · Logistic regression · High-dimensional analysis
- Questions that use it (3) — Pooling two groups · A two-sample statistic · When ordinary linear discrimination is not defined (answer)
- All 9 questions on Estimation and inference for the multivariate normal
See also
Invariance of the maximum likelihood estimator · Wishart distribution · Sampling law of the normal mean and scatter · Multivariate central limit theorem · The limit of Hotelling’s statistic · Null distribution of Hotelling’s · Two-sample Hotelling test · Likelihood of a multivariate normal sample