Rao–Blackwell theorem
Section of Rao–Blackwell and UMVU, part of Statistical inference
Rao–Blackwell theorem
| Type | Section |
| Taught in | Rao–Blackwell and UMVU |
| Part | Statistical inference |
Let T be sufficient for \theta, and let U be an unbiased estimator of \theta with finite variance. Define
W(T)=\mathbb{E}(U\mid T).
The Rao–Blackwell theorem says three things.
- W(T) does not depend on \theta. Sufficiency means the conditional distribution of the sample given T is free of \theta, and U is a function of the sample, so the conditional expectation is a function of T alone. It can be computed from the data.
- W is unbiased for \theta. The tower property gives \mathbb{E}\{W(T)\}=\mathbb{E}(U)=\theta.
- \operatorname{Var}\{W(T)\}\le\operatorname{Var}(U), with equality if and only if U is already a function of T. The conditional-variance formula is
\operatorname{Var}(U)=\operatorname{Var}\{\mathbb{E}(U\mid T)\}+\mathbb{E}\{\operatorname{Var}(U\mid T)\}=\operatorname{Var}\{W(T)\}+\mathbb{E}\{\operatorname{Var}(U\mid T)\}.
The second term is nonnegative, and it is zero precisely when U does not vary once T is known.
So any unbiased estimator that ignores a sufficient statistic can be improved by averaging it over that statistic. There is no point searching for a UMVU estimator outside the functions of T.
The same identity is why a sufficient statistic is enough for estimation of \theta and not enough for checking the model. \operatorname{Var}(U\mid T) is the variation T throws away. That variation does not help estimate \theta. It is exactly what a residual, or an ancillary statistic, is made of. Basu’s theorem says that under completeness this leftover is independent of T.
Used in
See also
Cramér–Rao lower bound · Fisher information · Score function · Uniformly minimum-variance unbiased estimator