Residuals in a GLM
Section of Paired comparisons, and how far a fit is from the data, part of Generalised linear models
Residuals in a GLM
| Type | Section |
| Taught in | Paired comparisons, and how far a fit is from the data |
| Part | Generalised linear models |
A coin with p=0.7 that lands heads did not commit an error of 0.3. A fair coin commits an “error” of 0.5 no matter which face appears, if error means y-\mu. And a count of 7 when you expected 2 is not the same failure as a count of 45 when you expected 50, even though both are five away. The likelihood of a Poisson 2 producing a 7 is not the likelihood of a Poisson 50 producing a 45.
The Pearson residual divides by the estimated standard deviation: r_P=\frac{y-\hat\mu}{\sqrt{\hat\phi\, V(\hat\mu)}}. For a binary observation with \hat p=0.7, \frac{1-0.7}{\sqrt{0.7\times 0.3}}=0.655, \qquad \frac{0-0.7}{\sqrt{0.7\times 0.3}}=-1.53. Landing tails is the larger Pearson residual. At \hat p=0.5 the two residuals are +1 and -1. For Poisson counts, with \phi=1, expecting 2 and seeing 7 gives (7-2)/\sqrt{2}=3.54. Expecting 50 and seeing 45 gives (45-50)/\sqrt{50}=-0.71. The unit is fairer. The sign is still symmetric: 55 instead of 45 would flip the sign and keep the magnitude, and a Poisson distribution is not symmetric.
The deviance residual takes one observation’s contribution d_i to D and attaches the sign of y_i-\hat\mu_i: r_D=\mathrm{sign}(y_i-\hat\mu_i)\sqrt{d_i}. At \hat p=0.7 the binary deviance residuals are +0.844 and -1.55. At \hat p=0.5 they are \pm 1.177, not \pm 1. For the Poisson, seeing 7 when expecting 2 gives \sqrt{2\bigl\{7\log(7/2)-(7-2)\bigr\}}=2.75, and seeing 45 when expecting 50 gives -0.72. The two deviations of five are no longer equal in magnitude once the likelihood, rather than the variance alone, is scoring them. That is the reason to prefer r_D when the distribution is skewed.
Plot either residual against the linear predictor, or against the fitted mean. For a continuous response you are looking for a flat, pattern-free scatter. For a binary response you will not get one. Every success lies on the curve 1-\hat p, and every failure lies on -\hat p, so the plot is two bands. A Poisson plot bands once for each integer that appears in the sample. The bands are the support of the distribution, not a missed covariate. Training your eye to ignore them is possible and unreliable. The probability-integral-transform residual, which turns a discrete fitted distribution back toward a uniform sample, is the next chapter.
Compute \exp(\hat\lambda_{\mathrm{Storm}}-\hat\lambda_{\mathrm{Sharks}}+\hat\alpha) from the table above, and convert the odds to a probability. Do the same for the Cowboys at home against the Knights.
Explain why the logit, probit, and complementary log-log cholesterol fits cannot be compared by a likelihood-ratio test, and what the three AIC values are allowed to say instead.
In the wafer analysis, the sequential p-value on
x2:x3:x4is 0.69. Say precisely which model is being compared with which, and why that is not the test of “are the three-factor interactions needed?”.For a Poisson mean of 50, the Pearson residual of y=55 is +0.71. Explain why the deviance residual of y=55 is not +0.72.
Used in
See also
Bradley–Terry model · Deviance · Three links, one cholesterol slope · Analysis of deviance · Saturated model · Residual deviance · Null deviance · Scaled deviance