Wilks’ theorem
Section of Generalised likelihood-ratio tests, part of Statistical inference
Wilks’ theorem
| Type | Section |
| Taught in | Generalised likelihood-ratio tests |
| Part | Statistical inference |
Suppose \Omega_0 is \Omega_1 with r smooth restrictions, the true parameter is an interior point of \Omega_0, and the regularity conditions for the asymptotic normality of maximum likelihood hold in the larger model. Then, when H_0 is true,
-2\log\Lambda\xrightarrow{d}\chi^2_r.
The degrees of freedom are the number of restrictions, not the number of observations. A test of one coefficient has r=1 and is compared with \chi^2_1. A test that J normal means are equal, with a common unknown variance, imposes J-1 restrictions and is compared with F, which is the exact finite-sample law; the chi-squared limit is what remains after the unknown variance is replaced by a consistent estimate and n grows.
The theorem is a large-sample statement. An exact pivot, when one exists, is preferable: the one-sample t statistic is a monotone function of the likelihood ratio and its null distribution is t_{n-1} for every n, not merely approximately \operatorname{N}(0,1). The chi-squared limit is what justifies referring -2\log\Lambda to \chi^2_r for a multinomial, a logistic regression, or any other regular model whose exact null distribution is intractable. It fails when the null puts a parameter on the boundary, the usual example being a variance component tested at zero. The printed tail probability is then conservative relative to the correct mixture of chi-squared laws.
A very small value of a chi-squared statistic is also informative, in the other direction. Pearson’s statistic on 3 degrees of freedom equal to 0.08 has left-tail probability \mathbb{P}(\chi^2_3\le 0.08)\approx 0.006, below the lower 5\% point 0.352. The counts match the model more closely than independent sampling would usually allow. One mechanism is pseudo-replication: the same experimental unit measured twice and entered as two independent observations, which shrinks the apparent noise. Another is that the counts were adjusted toward the fitted proportions. A goodness-of-fit test that looks only at the right tail will call this sample a remarkably good fit. The left tail is the check for a fit that is too good.
Used in
See also
Generalised likelihood-ratio test · Likelihood-ratio test · Pearson statistic