A shrunken sample mean
Let X_1,\ldots,X_n be a random sample with mean \mu and variance \sigma^2, and consider T_c=c\overline X for a constant 0<c\le1.
Find the bias, variance and mean squared error of T_c.
For which values of \mu does T_c with c<1 have a smaller mean squared error than \overline X? Why does this not make T_c a better estimator?
- \mathbb{E} T_c=c\mu, so the bias is (c-1)\mu. The variance is c^2\sigma^2/n. Hence
\operatorname{MSE}(T_c)=\frac{c^2\sigma^2}{n}+(1-c)^2\mu^2.
- \overline X is the case c=1, with mean squared error \sigma^2/n. T_c is better when
(1-c)^2\mu^2<(1-c^2)\frac{\sigma^2}{n},\qquad\text{that is}\qquad \mu^2<\frac{1+c}{1-c}\cdot\frac{\sigma^2}{n}.
So T_c wins when \mu is close to zero and loses when \mu is far from it. Since \mu is unknown, T_c is not better uniformly. It is better only if there is prior reason to think \mu is near 0, which is the idea behind shrinkage towards a prior mean in the Bayesian chapters.