Delta method
Section of Large-sample estimation, part of Statistical inference
Delta method
| Type | Section |
| Taught in | Large-sample estimation |
| Part | Statistical inference |
| Questions using it | 1 |
Let \sqrt n\,(T_n-\theta)\xrightarrow{d}\operatorname{N}(0,\xi^2), and let h have a derivative h'(\theta)\ne 0. Then
\sqrt n\,\bigl(h(T_n)-h(\theta)\bigr)\xrightarrow{d}\operatorname{N}\bigl(0,\xi^2\{h'(\theta)\}^2\bigr).
Expand h(T_n)=h(\theta)+(T_n-\theta)\{h'(\theta)+R_n\}. Consistency of T_n forces R_n\to 0 in probability when h' is continuous at \theta. Multiply by \sqrt n and apply the fact that a limit in distribution, multiplied by a quantity that converges in probability to a constant, converges to the constant times the limiting variable. The limiting variance picks up \{h'(\theta)\}^2.
This is why the maximum likelihood estimator of h(\theta) has asymptotic variance \{h'(\theta)\}^2/\{n\,i(\theta)\}. Invariance says the maximum likelihood estimator of h(\theta) is h(\widehat\theta). The delta method gives its variance. For a Poisson mean, \widehat\lambda=\overline X and h(\lambda)=e^{-\lambda}=\mathbb{P}(X=0),
\sqrt n\,(e^{-\overline X}-e^{-\lambda})\xrightarrow{d}\operatorname{N}\bigl(0,\lambda e^{-2\lambda}\bigr),
because h'(\lambda)=-e^{-\lambda} and \xi^2=\lambda. The estimated standard error is e^{-\widehat\lambda}\sqrt{\widehat\lambda/n}.
If h'(\theta)=0, the first term in the expansion vanishes and the rate changes. The next term is \tfrac12(T_n-\theta)^2 h''(\theta), so n\{h(T_n)-h(\theta)\} converges to a multiple of a chi-squared law rather than \sqrt n\{h(T_n)-h(\theta)\} converging to a normal law. Estimating a normal variance when the mean is the target of a different function is not this boundary case. Estimating \mu^2 at \mu=0 is: the derivative 2\mu is zero and \overline X^2 is of exact order 1/n, not 1/\sqrt n.
# Delta-method standard error for exp(-lambda) from a Poisson sample.
x <- c(0, 2, 1, 4, 1, 0, 3)
lam <- mean(x); n <- length(x)
exp(-lam) * sqrt(lam / n)Used in
- Taught in Large-sample estimation
- Questions that use it (1) — A probability from a Poisson mean
- All 2 questions on Large-sample estimation
See also
Asymptotic relative efficiency · Asymptotic normality of maximum likelihood · Maximum likelihood under the wrong model