Two-sample Hotelling T² test

Section of Estimation and inference for the multivariate normal, part of Statistical learning

Type	Section
Taught in	Estimation and inference for the multivariate normal
Part	Statistical learning

The one-sample test compares a mean with a fully specified vector. The more common question is whether two samples share the same mean. Let \mathbf{X}_i^{(1)}\sim\operatorname{N}_p(\boldsymbol{\mu}_1,\Sigma) for i=1,\ldots,n_1 and \mathbf{X}_i^{(2)}\sim\operatorname{N}_p(\boldsymbol{\mu}_2,\Sigma) for i=1,\ldots,n_2, with the two samples independent of each other and the observations independent inside each sample, and with one unknown covariance \Sigma. Test

H_0:\boldsymbol{\mu}_1-\boldsymbol{\mu}_2=\boldsymbol{\delta}_0 \qquad\text{against}\qquad H_1:\boldsymbol{\mu}_1-\boldsymbol{\mu}_2\neq\boldsymbol{\delta}_0,

usually with \boldsymbol{\delta}_0=\mathbf{0}. Every observation’s group is known before the statistic is computed. Predicting those labels would be a classification problem. This section estimates a difference of means.

Distribution of the mean difference

Main article: Distribution of the mean difference

The difference of the sample means is a linear image of the stacked data, so the linear-image proposition applies inside each sample and the covariances add. Stack the n_l observations of sample l as a vector of length n_l p. Independence makes its covariance block diagonal, with \Sigma repeated n_l times. The sample mean of sample l is therefore

\overline{\mathbf{X}}^{(l)}\sim\operatorname{N}_p(\boldsymbol{\mu}_l,\,\Sigma/n_l).

Under H_0 with \boldsymbol{\delta}_0=\mathbf{0}, independence of the two samples adds the covariances:

\overline{\mathbf{X}}^{(1)}-\overline{\mathbf{X}}^{(2)} \sim\operatorname{N}_p\Bigl(\mathbf{0},\,\Bigl(\frac1{n_1}+\frac1{n_2}\Bigr)\Sigma\Bigr) =\operatorname{N}_p\Bigl(\mathbf{0},\,\frac{n_1+n_2}{n_1 n_2}\Sigma\Bigr).

The factor 1/n_1+1/n_2 is (n_1+n_2)/(n_1 n_2). It is the sum of the sampling-variance coefficients of two independent sample means. A single sample mean contributes the coefficient 1/n.

The pooled covariance

Main article: Pooled covariance

The common \Sigma is estimated from both samples. Every observation informs that covariance, and the two means are not assumed equal under the alternative, so each sample is centred at its own mean. Estimating two means removes two degrees of freedom:

S=\frac{1}{n_1+n_2-2} \Biggl( \sum_{i=1}^{n_1}(\mathbf{x}_i^{(1)}-\bar{\mathbf{x}}^{(1)})(\mathbf{x}_i^{(1)}-\bar{\mathbf{x}}^{(1)})^\top +\sum_{i=1}^{n_2}(\mathbf{x}_i^{(2)}-\bar{\mathbf{x}}^{(2)})(\mathbf{x}_i^{(2)}-\bar{\mathbf{x}}^{(2)})^\top \Biggr).

This is the pooled formula with G=2. The Wishart statement applied to the pooled residuals gives

(n_1+n_2-2)S\sim W_p(\Sigma,\,n_1+n_2-2),

independent of both sample means and therefore of their difference. The one-sample passage from a normal vector and an independent Wishart matrix to an F statistic used \sqrt{n}(\overline{\mathbf{X}}-\boldsymbol{\mu}_0) and n-1 degrees of freedom. Replace those ingredients by the mean-difference law and by n_1+n_2-2 degrees of freedom. The resulting statistic is

T^2=\frac{n_1 n_2}{n_1+n_2} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr)^\top S^{-1} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr),

and it follows the T^2 law on n_1+n_2-2 degrees of freedom. The same rescaling as in the one-sample case produces

\frac{n_1+n_2-p-1}{(n_1+n_2-2)\,p}\,T^2 \sim F_{p,\,n_1+n_2-p-1}.

Theorem 1 (Two-sample Hotelling test) Under the common-covariance normal model above, reject H_0:\boldsymbol{\mu}_1=\boldsymbol{\mu}_2 at level \alpha when

\frac{n_1+n_2-p-1}{(n_1+n_2-2)\,p}\,T^2 >F_{\alpha;\,p,\,n_1+n_2-p-1}.

The second degree of freedom is n_1+n_2-p-1. It is the Wishart degrees of freedom n_1+n_2-2, reduced by the dimension in the same way that n-1 became n-p in the one-sample F.

Article: Two-sample Hotelling test →

Confidence statements

Main article: Simultaneous confidence statements

Invert the test, as with the one-sample region. The values of \boldsymbol{\delta}=\boldsymbol{\mu}_1-\boldsymbol{\mu}_2 that the level-\alpha test would not reject form the confidence set

\bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}-\boldsymbol{\delta}\bigr)^\top S^{-1} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}-\boldsymbol{\delta}\bigr) \le\frac{n_1+n_2}{n_1 n_2}\,T^2_{\alpha;\,p,\,n_1+n_2-p-1},

where

T^2_{\alpha;\,p,\,n_1+n_2-p-1} =\frac{(n_1+n_2-2)\,p}{n_1+n_2-p-1}\,F_{\alpha;\,p,\,n_1+n_2-p-1}.

The same critical value yields simultaneous intervals for every linear functional \mathbf{a}^\top\boldsymbol{\delta}, \mathbf{a}\in\mathbb{R}^p, and not only for the p coordinates:

\mathbf{a}^\top\bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr) \pm \sqrt{\mathbf{a}^\top S\mathbf{a}\cdot\frac{n_1+n_2}{n_1 n_2}\,T^2_{\alpha;\,p,\,n_1+n_2-p-1}}.

A coordinate interval is the case in which \mathbf{a} is a unit vector along one axis. The half-width uses the T^2 critical value rather than a single two-sample t critical value, because the intervals are required to cover \mathbf{a}^\top\boldsymbol{\delta} simultaneously for every \mathbf{a}, including the direction in which the two sample means differ most. Separate t intervals each control one coordinate and do not control a search over directions, so the simultaneous intervals are wider.

Unequal covariance matrices

Main article: Unequal covariance matrices

Pooling is justified by the common-covariance model. Equal group sizes do not make that assumption true, and they do not restore the exact finite-sample F distribution under arbitrary heteroscedasticity. Large-sample robustness of the pooled procedure is a more limited conclusion. With unequal covariances the problem is the multivariate Behrens–Fisher problem, even when the group sizes happen to match.

One approximation, due to Krishnamoorthy and Yu, estimates the covariance of the mean difference directly. Define \widetilde S_l=S_l/n_l and \widetilde S=\widetilde S_1+\widetilde S_2. The statistic is

T^2=\bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr)^\top \widetilde S^{-1} \bigl(\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\bigr).

For independent normal samples with n_l>p and positive definite population covariances, an F approximation is

\frac{\nu-p+1}{\nu p}\,T^2\sim F_{p,\,\nu-p+1},

with denominator degrees of freedom \nu-p+1, where

\nu=\frac{p+p^2} {\displaystyle\sum_{l=1}^{2} \dfrac{\operatorname{tr}\bigl[(\widetilde S_l\widetilde S^{-1})^2\bigr] +\bigl[\operatorname{tr}(\widetilde S_l\widetilde S^{-1})\bigr]^2}{n_l-1}}.

The traces compare each sample’s share of the total covariance estimate. The divisors n_l-1 are the covariance degrees of freedom. For p=1, put v_l=s_l^2/n_l. The formula reduces to

\nu=\frac{(v_1+v_2)^2}{v_1^2/(n_1-1)+v_2^2/(n_2-1)},

which is Welch’s degrees of freedom. The reference law remains approximate. Comparisons of several mean vectors, with a common covariance, continue as a multivariate analysis of variance: the pooled within-group matrix is the error stratum, and the between-group matrix of weighted outer products of group-mean deviations is compared with it by a further Wishart ratio.

Two-sample Hotelling T² test

Distribution of the mean difference

The pooled covariance

Confidence statements

Unequal covariance matrices

Used in

See also