Q34. Deviance and model adequacy
[5 marks]
A Poisson regression gives:
Null deviance: 410.0 on 79 degrees of freedom
Residual deviance: 310.0 on 77 degrees of freedom(a) Deduce the number of observations and the number of estimated regression coefficients, including the intercept.
(b) Explain which two models are being compared by the null deviance, and which two are being compared by the residual deviance.
(c) Test whether adding the predictors improves on an intercept-only model. You may use \chi^2_{2,0.95}=5.99.
(d) A researcher concludes: “The predictors are significant, so the model fits adequately.” Assess this conclusion using the output.
For this question, assume the usual large-sample deviance approximations are appropriate.
Definitions. The saturated model fits a separate mean to each observation. Deviance is 2(\ell_{\mathrm{sat}}-\ell_{\mathrm{fit}}), using maximised log-likelihoods. Null deviance uses an intercept-only fit; residual deviance uses the stated predictor model. A difference in deviances compares nested models fitted to the same data. Let n count observations and q count estimated regression coefficients, including the intercept.
Assumptions. As stipulated in the question, the large-sample chi-squared deviance approximations are appropriate. Test at 5%, using models fitted to the same observations. The goodness-of-fit calibration also relies on the specified Poisson variance model.
(a) Numbers of observations and coefficients. The intercept-only model has one coefficient, so its residual degrees of freedom give n-1=79\quad\Longrightarrow\quad\boxed{n=80}. If the fitted model has q regression coefficients including the intercept, n-q=77\quad\Longrightarrow\quad\boxed{q=3}.
(b) What the deviances compare. Write \ell_{\mathrm{sat}}, \ell_0, and \ell_1 for the maximised log-likelihoods of the saturated, intercept-only, and fitted predictor models, respectively. Then D_{\mathrm{null}}=2(\ell_{\mathrm{sat}}-\ell_0)=410, \qquad D_{\mathrm{res}}=2(\ell_{\mathrm{sat}}-\ell_1)=310. The saturated model fits a separate mean to each observation. Null deviance compares it with the intercept-only model; residual deviance compares it with the model containing the predictors.
(c) Test whether adding the predictors helps. With two predictor coefficients, test H_0:\beta_1=\beta_2=0, \qquad H_1:\text{at least one of these coefficients is nonzero}. The likelihood-ratio statistic is G^2=2(\ell_1-\ell_0) =D_{\mathrm{null}}-D_{\mathrm{res}} =410-310=100. Under H_0, use the stipulated approximation G^2\sim\chi^2_2. Since 100>5.99, reject H_0 at the 5\% level. Conclusion: Under the Poisson assumptions, adding the predictors significantly improves fit compared with the intercept-only model.
(d) Does this establish adequate fit? No. Under an adequate fitted model and the stated approximation, the residual deviance is compared with \chi^2_{77}. The observed deviance 310 exceeds the 95th percentile \chi^2_{77,0.95}\approx98.48, so reject adequacy at the 5\% level. It is also large relative to the degrees of freedom: \frac{D_{\mathrm{res}}}{77}=\frac{310}{77}\approx4.03. This indicates lack of fit, despite the improvement in part (c). Possible causes include an incorrect mean structure or overdispersion; these two numbers alone do not identify the cause. Model inadequacy can also undermine the reliability of nominal model-based tests and standard errors, so the part (c) conclusion is conditional on its assumptions.
Key result. A model can be significantly better than a poor baseline and still fit the data badly. Improvement is not adequacy.