Residual deviance

Topic of Paired comparisons, and how far a fit is from the data, part of Generalised linear models

Residual deviance

Type Topic
Taught in Paired comparisons, and how far a fit is from the data
Part Generalised linear models
Questions using it 3

The deviance of a fitted model, D=2\{\ell(\mathbf{y};\mathbf{y})-\ell(\hat{\boldsymbol\mu};\mathbf{y})\}, printed by R with its residual degrees of freedom. The difference in residual deviance between nested models is a likelihood-ratio statistic.

In Paired comparisons, and how far a fit is from the data

From Paired comparisons, and how far a fit is from the data

The deviance is twice the drop in log-likelihood from the saturated fit to the model you actually fitted: D({\boldsymbol{y}};\hat{\boldsymbol{\mu}})=2\bigl\{\ell({\boldsymbol{y}};{\boldsymbol{y}})-\ell(\hat{\boldsymbol{\mu}};{\boldsymbol{y}})\bigr\}, up to the dispersion, which scales it. On the observation scale a residual of 5 is not the same thing for a Poisson mean of 2 and a Poisson mean of 50. The likelihood already knows that. For the families we have been using, the deviance contributions add up as follows. For the normal, \phi=\sigma^2, D=\sum_{i=1}^n(y_i-\hat\mu_i)^2, the residual sum of squares, because the factor of 2 cancels the \tfrac12 in the normal log-density. For the Poisson, D=2\sum_{i=1}^n\left\{y_i\log\frac{y_i}{\hat\mu_i}-(y_i-\hat\mu_i)\right\}. For a binomial count of y_i successes out of m_i, with fitted mean count m_i\hat p_i, D=2\sum_{i=1}^n\left\{y_i\log\frac{y_i}{m_i\hat p_i}+(m_i-y_i)\log\frac{m_i-y_i}{m_i(1-\hat p_i)}\right\}. The gamma deviance, in the mean-and-variance parameterisation \mathrm{Var}=\phi\mu^2, is D=2\sum_{i=1}^n\left\{-\log\frac{y_i}{\hat\mu_i}+\frac{y_i-\hat\mu_i}{\hat\mu_i}\right\}. These are not interchangable across families. A smaller deviance is a closer fit only inside one family. Two non-nested models, such as a logit and a probit, have no likelihood-ratio test. AIC is the comparison that remains: deviance plus twice the number of parameters, up to the usual constants.

In Paired comparisons, and how far a fit is from the data, continued

From Paired comparisons, and how far a fit is from the data

The three-factor model has residual deviance 0.0084 on 1 degree of freedom. The two-factor model has residual deviance 0.0339 on 5 degrees of freedom. The difference is 0.0255 on 4 and 1 degrees of freedom: F=0.76,\qquad p=0.68. The three-factor terms can go. Comparing main effects with the two-factor model gives F=2.23 on 6 and 5 degrees of freedom, p=0.20. The pairwise products can go as well. Among the four main effects, x_4 has t=-1.10. Dropping it leaves

(Intercept)  5.4155
x1+          0.1211
x2+         -0.2998
x3+          0.1824

so the fitted mean, with each x coded +1 at its high level, is \hat\mu=\exp(5.4155+0.1211\,x_1-0.2998\,x_2+0.1824\,x_3). A quoted value of the x_2 coefficient is -0.2981. The coefficient table, which is the fit, says -0.2998. We use the table.

Used in

See also

Deviance · Null deviance · Scaled deviance · Saturated model · Deviance residual · Degrees of freedom · Likelihood-ratio test · Bradley–Terry model · Three links, one cholesterol slope

Glossary and index