Hotelling’s T² as a likelihood-ratio test
Section of Estimation and inference for the multivariate normal, part of Statistical learning
Hotelling’s T² as a likelihood-ratio test
| Type | Section |
| Taught in | Estimation and inference for the multivariate normal |
| Part | Statistical learning |
Take n independent observations from \operatorname{N}_p(\boldsymbol{\mu},\Sigma), with both parameters unknown, and test
H_0:\boldsymbol{\mu}=\boldsymbol{\mu}_0 \qquad\text{against}\qquad H_1:\boldsymbol{\mu}\neq\boldsymbol{\mu}_0.
The unrestricted fit uses \bar{\mathbf{x}} and A/n. Under the null, the mean is fixed and only the covariance can adapt. The likelihood-ratio statistic compares those two best fits. The construction does not by itself establish uniform optimality against every multivariate alternative. Compare the maximised likelihood with \boldsymbol{\mu} fixed at \boldsymbol{\mu}_0 to the maximised likelihood with \boldsymbol{\mu} free:
\lambda=\frac{\max_\Sigma L(\boldsymbol{\mu}_0,\Sigma)}{\max_{\boldsymbol{\mu},\Sigma}L(\boldsymbol{\mu},\Sigma)}\in[0,1].
The numerator maximises over a subset of the denominator’s parameter space, so the ratio cannot exceed one. If it is small, imposing the proposed mean costs a great deal of fit.
Maximisation under H_0. Under H_0 only \Sigma is free. The derivative that produced \widehat\Sigma=A/n measures residuals from the centring point in use. With residuals measured from \boldsymbol{\mu}_0 rather than from a free mean, the same derivative gives
\widehat\Sigma_\omega=\frac1n\sum_{i=1}^n(\mathbf{x}_i-\boldsymbol{\mu}_0)(\mathbf{x}_i-\boldsymbol{\mu}_0)^\top.
Split those residuals at \bar{\mathbf{x}}:
\begin{aligned} \widehat\Sigma_\omega &=\frac1n\sum_{i=1}^n\bigl[(\mathbf{x}_i-\bar{\mathbf{x}})+(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)\bigr] \bigl[(\mathbf{x}_i-\bar{\mathbf{x}})+(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)\bigr]^\top\\ &=\frac1n\Bigl[A+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top\Bigr]. \end{aligned}
The cross terms vanish because \sum_i(\mathbf{x}_i-\bar{\mathbf{x}})=\mathbf{0}. The constrained covariance estimate is A/n plus the outer product of \bar{\mathbf{x}}-\boldsymbol{\mu}_0. The added term is positive semi-definite, and it is zero only when the sample mean equals the null mean.
Maximisation without restriction. With both parameters free, \widehat{\boldsymbol{\mu}}_\Omega=\bar{\mathbf{x}} and \widehat\Sigma_\Omega=A/n. The denominator of \lambda uses those two estimators.
The ratio. Substitute the two maximisers. The powers of 2\pi agree. The exponents also agree, for a reason that separates from the determinant algebra: at each maximiser the quadratic form in the exponent equals np. A scalar equals its own trace, and the trace is cyclic, so \mathbf{v}^\top M^{-1}\mathbf{v}=\operatorname{tr}(M^{-1}\mathbf{v}\mathbf{v}^\top). Apply that identity to the constrained exponent:
\begin{aligned} \sum_{i=1}^n(\mathbf{x}_i-\boldsymbol{\mu}_0)^\top\widehat\Sigma_\omega^{-1}(\mathbf{x}_i-\boldsymbol{\mu}_0) &=\operatorname{tr}\Bigl(\widehat\Sigma_\omega^{-1}\sum_{i=1}^n(\mathbf{x}_i-\boldsymbol{\mu}_0)(\mathbf{x}_i-\boldsymbol{\mu}_0)^\top\Bigr)\\ &=\operatorname{tr}\bigl(\widehat\Sigma_\omega^{-1}\cdot n\widehat\Sigma_\omega\bigr) =n\operatorname{tr}(I_p)=np. \end{aligned}
At the constrained maximiser, n\widehat\Sigma_\omega equals \sum_i(\mathbf{x}_i-\boldsymbol{\mu}_0)(\mathbf{x}_i-\boldsymbol{\mu}_0)^\top. The same steps with \widehat\Sigma_\Omega and residuals from \bar{\mathbf{x}} also give np, because n\widehat\Sigma_\Omega=A. Both exponential factors equal e^{-np/2} and cancel. Normality was used to write the likelihood. The cancellation itself is this trace identity at the two critical points. The likelihood ratio is a ratio of determinants,
\lambda=\frac{|\widehat\Sigma_\Omega|^{n/2}}{|\widehat\Sigma_\omega|^{n/2}} =\frac{|A|^{n/2}}{\bigl|A+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top\bigr|^{n/2}}.
A determinant identity. For conformable blocks with B invertible,
\begin{vmatrix}B&C\\ D&E\end{vmatrix} =\begin{vmatrix}B&C\\ 0&E-DB^{-1}C\end{vmatrix} =|B|\,|E-DB^{-1}C|.
The first equality subtracts DB^{-1} times the first block row from the second, which does not change the determinant. The same determinant, eliminated on the lower-right block instead, has the companion form |B|\,|E-DB^{-1}C|=|E|\,|B-CE^{-1}D|. Apply the identity with B=A, C=-\sqrt{n}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0), D=\sqrt{n}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top and E=1:
\begin{vmatrix} A & -\sqrt{n}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)\\ \sqrt{n}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top & 1 \end{vmatrix} =|A|\Bigl(1+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top A^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)\Bigr).
With E=1 the companion evaluation is |A+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top|, because -CE^{-1}D=+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top. The two evaluations therefore give
\bigl|A+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top\bigr| =|A|\Bigl(1+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top A^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)\Bigr).
This comparison is algebraic. Raise \lambda to the power 2/n:
\lambda^{2/n} =\frac{1}{1+n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top A^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)} =\frac{1}{1+t^2/(n-1)},
where A=(n-1)S has been used to name the quadratic form
t^2=n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top S^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0) =(n-1)\,n(\bar{\mathbf{x}}-\boldsymbol{\mu}_0)^\top A^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu}_0).
The two expressions agree because S^{-1}=(n-1)A^{-1}. Thus \lambda is a strictly decreasing function of t^2. Rejecting for small \lambda is the same test as rejecting for large t^2. The block determinant converts the ratio of covariance determinants into that quadratic form, and the conversion does not use normality. Normality enters when the threshold is given a probability. The statistic t^2 is n times the squared Mahalanobis distance from \bar{\mathbf{x}} to \boldsymbol{\mu}_0, computed with S. When p=1, S reduces to s^2 and t^2 is the square of the univariate t statistic.
Used in
See also
Invariance of the maximum likelihood estimator · Wishart distribution · Sampling law of the normal mean and scatter · Multivariate central limit theorem · The limit of Hotelling’s statistic · Null distribution of Hotelling’s · Two-sample Hotelling test · Likelihood of a multivariate normal sample