Loss and risk
Section of Estimation, part of Statistical inference
Loss and risk
| Type | Section |
| Taught in | Estimation |
| Part | Statistical inference |
Unbiasedness is one restriction that stops an estimator from cheating. A loss function is the scale on which the cheating would have been measured. A loss \ell(\theta,t) is nonnegative and zero when t=\theta. The risk is the expected loss,
R(\theta,T)=\mathbb{E}_\theta\bigl\{\ell(\theta,T)\bigr\}.
Squared error, \ell(\theta,t)=(\theta-t)^2, has risk equal to the mean squared error. Absolute error, \ell(\theta,t)=|\theta-t|, penalises a large miss in proportion to its size rather than its square. For a sample from a distribution symmetric about \theta, the estimator that minimises \sum_i|x_i-t| is the sample median, and the estimator that minimises \sum_i(x_i-t)^2 is the sample mean. Letting the power on the absolute deviation tend to infinity moves the minimiser to the mid-range, halfway between the smallest and largest observations. The three estimators answer three different losses. None of them is the risk minimiser for every \theta at once.
No estimator has smallest risk at every \theta, except in the trivial parameter space that contains only one point. If \theta_0 is the only possible value, the constant estimator T=\theta_0 has risk zero and nothing beats it. If some other value is also possible, that constant has positive risk there, and the constant at the other value beats it at that other value. Restricting the competition, to unbiased estimators or to equivariant ones, is what makes “best” mean something. The Cramér–Rao bound is the result of restricting to unbiased estimators. Squared-error risk without that restriction has no uniform winner.
Used in
- Taught in Estimation
- All 6 questions on Estimation
See also
Estimator · Method of moments · Sample correlation coefficient · Likelihood function · Maximum likelihood · Invariance of maximum likelihood · Unbiased estimator · Mean squared error