Mean squared error
Definition taught in Estimation, part of Statistical inference
Mean squared error
| Type | Definition |
| Taught in | Estimation |
| Part | Statistical inference |
| Worked examples | 3 |
| Questions using it | 3 |
The mean squared error of an estimator T of \theta is \operatorname{MSE}(T)=\mathbb{E}(T-\theta)^2. It splits as
\mathbb{E}(T-\theta)^2=\operatorname{Var}(T)+(\mathbb{E} T-\theta)^2.
Examples
Two variance estimators
Worked example from Estimation
Let X_1,\ldots,X_n be a normal random sample with n\ge 2.
Question. Compare the mean squared errors of s^2 and \widehat\sigma^2=((n-1)/n)s^2 as estimators of \sigma^2.
Solution. By the normal sample theorem, V=(n-1)s^2/\sigma^2 is \chi^2_{n-1}, so \operatorname{Var} V=2(n-1) and
\operatorname{Var}(s^2)=\frac{\sigma^4}{(n-1)^2}\,2(n-1)=\frac{2\sigma^4}{n-1}.
s^2 is unbiased, so this is its mean squared error. For \widehat\sigma^2, the variance is ((n-1)/n)^2 times as large and the bias is -\sigma^2/n:
\operatorname{MSE}(\widehat\sigma^2)=\frac{(n-1)^2}{n^2}\cdot\frac{2\sigma^4}{n-1}+\frac{\sigma^4}{n^2}=\frac{(2n-1)\sigma^4}{n^2}.
Since (2n-1)(n-1)=2n^2-3n+1<2n^2, the biased estimator has the smaller mean squared error for every n and every \sigma^2. With n=10 the two are 0.222\,\sigma^4 and 0.190\,\sigma^4. The unbiased estimator is preferred for other reasons: it is the one whose distribution gives the exact t and chi-squared pivots.
Two estimators of a uniform endpoint
Worked example from Estimation
Let X_1,\ldots,X_n be a random sample from the uniform distribution on (0,\theta), and let M=\max_i X_i.
Question. Compare the mean squared errors of the moment estimator 2\overline X and the maximum likelihood estimator M.
Solution. 2\overline X is unbiased, and \operatorname{Var} X_i=\theta^2/12, so
\operatorname{MSE}(2\overline X)=4\cdot\frac{\theta^2}{12n}=\frac{\theta^2}{3n}.
For the maximum, \mathbb{P}(M\le m)=(m/\theta)^n on (0,\theta), so M has density nm^{n-1}/\theta^n. Then \mathbb{E} M=n\theta/(n+1) and \mathbb{E} M^2=n\theta^2/(n+2), and
\operatorname{MSE}(M)=\mathbb{E} M^2-2\theta\mathbb{E} M+\theta^2=\frac{2\theta^2}{(n+1)(n+2)}.
The maximum is biased low, because it can never exceed \theta, but its mean squared error falls like 1/n^2 rather than 1/n. With n=10 the two are 0.033\,\theta^2 and 0.015\,\theta^2. The rescaled maximum (n+1)M/n is unbiased and has mean squared error \theta^2/\{n(n+2)\}, smaller still.
A shrunken sample mean
Question page: A shrunken sample mean
Let X_1,\ldots,X_n be a random sample with mean \mu and variance \sigma^2, and consider T_c=c\overline X for a constant 0<c\le1.
Find the bias, variance and mean squared error of T_c.
For which values of \mu does T_c with c<1 have a smaller mean squared error than \overline X? Why does this not make T_c a better estimator?
- \mathbb{E} T_c=c\mu, so the bias is (c-1)\mu. The variance is c^2\sigma^2/n. Hence
\operatorname{MSE}(T_c)=\frac{c^2\sigma^2}{n}+(1-c)^2\mu^2.
- \overline X is the case c=1, with mean squared error \sigma^2/n. T_c is better when
(1-c)^2\mu^2<(1-c^2)\frac{\sigma^2}{n},\qquad\text{that is}\qquad \mu^2<\frac{1+c}{1-c}\cdot\frac{\sigma^2}{n}.
So T_c wins when \mu is close to zero and loses when \mu is far from it. Since \mu is unknown, T_c is not better uniformly. It is better only if there is prior reason to think \mu is near 0, which is the idea behind shrinkage towards a prior mean in the Bayesian chapters.
Used in
- Taught in Estimation, where the derivation and examples are.
- Also used in Statistical inference
- Questions that use it (3) — A Poisson mean · A shrunken sample mean · Posterior mean under squared-error loss
- All 5 questions on Estimation
See also
Estimator · Method of moments · Sample correlation coefficient · Likelihood function · Maximum likelihood · Invariance of maximum likelihood · Unbiased estimator · Consistent estimator