Wald test
Topic of Logistic regression, part of Statistical learning · Also in Confidence intervals
Wald test
| Type | Topic |
| Taught in | Logistic regression · Confidence intervals |
| Part | Statistical learning · Statistical inference |
| Questions using it | 5 |
A test built from an estimate and its standard error: z=(\hat\theta-\theta_0)/\widehat{\operatorname{se}}(\hat\theta), referred to \mathrm{N}(0,1). Inverting it gives the Wald interval \hat\theta\pm z_{\alpha/2}\,\widehat{\operatorname{se}}(\hat\theta). It is the z value that R prints beside each GLM coefficient.
In Logistic regression
From Logistic regression
At a regular finite logistic maximum likelihood estimate, invert X^\top W X and take square roots of its diagonal entries to obtain asymptotic standard errors. The Wald statistic divides an estimated coefficient by its standard error. This is an approximation justified by sampling behaviour under the model, not a guarantee supplied by the numerical inverse. The Wald interval and the deviance comparison both assume that a finite maximum was attained. Under complete separation the coefficients diverge and those standard errors are not meaningful.
In Confidence intervals
From Confidence intervals
Let X\sim\operatorname{Bin}(n,p) and \widehat p=X/n. An exact interval is awkward because the binomial law is discrete. For large n the central limit theorem gives
\frac{\widehat p-p}{\sqrt{p(1-p)/n}}\approx\operatorname{N}(0,1),
so \mathbb{P}(-z_{1-\alpha/2}<(\widehat p-p)/\sqrt{p(1-p)/n}<z_{1-\alpha/2})\approx1-\alpha. Solving for p leaves p inside the standard error. For large n, \widehat p is close to p, and replacing the unknown standard error by \sqrt{\widehat p(1-\widehat p)/n} produces an interval whose ends depend only on the data.
Definition 1 (Wald interval) For X\sim\operatorname{Bin}(n,p) with \widehat p=X/n, the Wald interval with approximate confidence 1-\alpha is
\widehat p\pm z_{1-\alpha/2}\sqrt{\frac{\widehat p(1-\widehat p)}{n}}.
It is an approximation. It behaves badly when n is small or \widehat p is near 0 or 1, because the estimated standard error is then a poor stand-in and the normal shape has not set in. At \widehat p=0 it collapses to a single point. A common rule is to use it only when n\widehat p and n(1-\widehat p) are both at least 5. It also models the sample as independent draws: a poll with refusals or untruthful answers has errors no interval formula can see.
For two independent samples the same approximation applies to \widehat p_1-\widehat p_2, whose variance is the sum of the two variances. The interval is
\widehat p_1-\widehat p_2\pm z_{1-\alpha/2}\sqrt{\frac{\widehat p_1(1-\widehat p_1)}{n_1}+\frac{\widehat p_2(1-\widehat p_2)}{n_2}}.
Example 1 (Nightmares in two groups) Of 160 men, 55 report frequent nightmares; of 192 women, 60 do.
Question. Give an approximate 95% confidence interval for the difference p_1-p_2 in the population proportions, and interpret it.
Solution. \widehat p_1=55/160=0.344 and \widehat p_2=60/192=0.313, a difference of 0.031. The standard error is
\sqrt{\frac{0.344\times0.656}{160}+\frac{0.313\times0.688}{192}}=0.0503,
so the interval is 0.031\pm1.96\times0.0503=(-0.067,\ 0.130). It contains 0. The data are consistent with no difference between men and women, and also with a difference of several percentage points either way.
Used in
- Taught in Logistic regression · Confidence intervals
- Questions that use it (5) — Reading the deviance · Q37 (answer) · Q41 (answer) · Q47 · Question 21 (answer)
- All 7 questions on Logistic regression
See also
Standard error · Generalised linear model · Binary logistic regression · A finite stationary point is a global maximiser · From a linear model to class probabilities · Logistic and probit models · Likelihood, score, and fitting · Score and Hessian of logistic regression