Uniformly most powerful unbiased test

Section of Most powerful tests, part of Statistical inference

Type	Section
Taught in	Most powerful tests
Part	Statistical inference

Now let H_1:\mu\ne\mu_0, still with known normal variance. Against an alternative \mu_1>\mu_0 the most powerful test rejects for large \overline X. Against \mu_1<\mu_0 it rejects for small \overline X. Those are different regions. No single region is most powerful against both. A uniformly most powerful test of the two-sided hypothesis does not exist.

The usual two-sided z test rejects when |\overline X-\mu_0|>z_{1-\alpha/2}\,\sigma/\sqrt n. It is not most powerful against a one-sided alternative: the one-sided test of the same size puts all of its critical probability on the side where that alternative lives, and therefore has higher power there. The two-sided test spends half its type I error on a side the one-sided alternative says cannot occur.

What the two-sided test does have is unbiasedness. A test is unbiased when its power is at least \alpha at every alternative. The one-sided test fails that requirement for the two-sided problem: against an alternative on the other side its power is less than \alpha. Among unbiased tests, the equal-tailed region is uniformly most powerful. It is the uniformly most powerful unbiased test of H_0:\mu=\mu_0 against H_1:\mu\ne\mu_0. The threshold is

k=z_{1-\alpha/2}\,\frac{\sigma}{\sqrt n},

and the critical region is |\overline X-\mu_0|>k.

The same split explains the t test when \sigma is unknown. Replacing \sigma by s in the likelihood ratio, using the joint normal likelihood of the sample, reduces a one-sided test to a one-sided t_{n-1} critical value and a two-sided test to the equal-tailed t critical value. The unknown variance is estimated by the residual sum of squares, which is independent of \overline X in a normal sample, and the ratio is the t pivot already used for confidence intervals. The duality is exact: the two-sided level-\alpha test accepts \mu_0 if and only if the level 1-\alpha interval contains \mu_0.

# Level 0.05 critical value for a two-sided t test, n = 42.
qt(0.975, df = 41)     # 2.020
# A one-sided test at the same level uses the 0.95 quantile, which is smaller.
qt(0.95, df = 41)      # 1.683

The one-sided quantile is smaller because the whole type I error sits in one tail. Using it for a two-sided scientific question, after seeing the sign of \overline X-\mu_0, is the one-sided test with a direction chosen from the data. Its type I error is then about 2\alpha, not \alpha.

Key idea

For two simple hypotheses the most powerful test of a given size rejects when the likelihood under the alternative is large relative to the likelihood under the null. If every alternative in a one-sided family produces the same region, the test is uniformly most powerful. A two-sided normal mean has no such test. The equal-tailed test is the most powerful test inside the smaller class of unbiased tests.

Used in

Taught in Most powerful tests
All 2 questions on Most powerful tests

Used in

See also