Uniformly most powerful test
Section of Most powerful tests, part of Statistical inference
Uniformly most powerful test
| Type | Section |
| Taught in | Most powerful tests |
| Part | Statistical inference |
| Questions using it | 1 |
Let the alternative be composite, H_1:\theta\in\Omega_1. A test is uniformly most powerful at level \alpha if its size is at most \alpha and, for every \theta_1\in\Omega_1, it is most powerful against the simple alternative \theta=\theta_1.
For a normal mean with known variance and H_1:\mu>\mu_0, pick any \mu_1>\mu_0. The Neyman–Pearson test of \mu_0 against that \mu_1 rejects for \overline X>k with the same k=\mu_0+z_{1-\alpha}\,\sigma/\sqrt n, whatever \mu_1 was. The critical region does not depend on which alternative was used to build the ratio. It is therefore most powerful against every \mu_1>\mu_0, and the one-sided z test is uniformly most powerful.
The power at a particular \mu_1 is
\mathbb{P}_{\mu_1}(\overline X>k)=\Phi\Bigl(-z_{1-\alpha}+\frac{\mu_1-\mu_0}{\sigma/\sqrt n}\Bigr).
It increases with (\mu_1-\mu_0)\sqrt n/\sigma. A test can be uniformly most powerful and still have poor power if n is small or \sigma is large. Uniformity is a comparison with other tests of the same size, not a promise that the power is high.
The one-sided Poisson test has the same structure. For X_j independent Poisson(\lambda) and H_0:\lambda=\lambda_0 against H_1:\lambda>\lambda_0, the likelihood ratio is large when \sum X_j is large, for every \lambda_1>\lambda_0. The uniformly most powerful test rejects when \sum X_j\ge d, with d the smallest integer whose null probability \mathbb{P}\{\operatorname{Poisson}(n\lambda_0)\ge d\} does not exceed \alpha.
A t test of the same mean, which replaces the Poisson variance by the sample variance, is a different procedure. When the data are Poisson the likelihood-ratio test has the higher power. When they are not, the Poisson test can have a type I error far from \alpha, while the t statistic still has an approximate t null distribution under a finite variance. The lemma’s optimality is inside the model used to write the likelihood. It does not survive an arbitrary change of that model. The large-sample comparison of the two procedures is on Large-sample estimation.
A rule that is not a function of a statistic chosen before seeing the data is not in this competition at all. Rejecting when one particular patient’s improvement exceeds a threshold picked after looking at who improved can have any type I error one wants to claim, because the region was fitted to the sample. The power calculation for the one-sided normal test starts from a region R that does not depend on which sample was drawn.
Used in
- Taught in Most powerful tests
- Questions that use it (1) — Why the two-sided test is not UMP (answer)
- All 2 questions on Most powerful tests
See also
Neyman–Pearson lemma · Uniformly most powerful unbiased test · Neyman–Pearson approach