Analysis of deviance

Section of Paired comparisons, and how far a fit is from the data, part of Generalised linear models

Type	Section
Taught in	Paired comparisons, and how far a fit is from the data
Part	Generalised linear models

For nested models the difference in scaled deviance, deviance divided by \phi, is referred to a chi-squared law if \phi is known, and to an F law if \phi is estimated from the same data. That is the GLM version of ANOVA, and anova on a glm object is doing it even though the name still says ANOVA. If the columns of {\color{blue}\boldsymbol{X}} are not orthogonal, the order of entry matters. A sequential p-value is the test of dropping that term while keeping everything listed above it, and nothing listed below it.

Myers and Montgomery’s wafer experiment is a 2^4 factorial with no replication: sixteen runs, resistivity, which is positive and right-skewed. The family is gamma and the link is log, not the canonical inverse, because no corner of the design should send resistivity to infinity. A four-factor interaction would use sixteen parameters and fit the sample perfectly. The largest model we can actually look at stops at the three-factor interactions:

need <- function(pkg) {
  if (!requireNamespace(pkg, quietly = TRUE)) {
    install.packages(pkg, repos = "https://cloud.r-project.org")
  }
  suppressPackageStartupMessages(library(pkg, character.only = TRUE))
}
need("faraway")
data(wafer, package = "faraway")
wafer$y <- wafer$resist
fit3 <- glm(y ~ x1 * x2 * x3 * x4 - x1:x2:x3:x4,
            data = wafer, family = Gamma(link = "log"))
fit2 <- glm(y ~ (x1 + x2 + x3 + x4)^2,
            data = wafer, family = Gamma(link = "log"))
anova(fit2, fit3, test = "F")
fit1 <- glm(y ~ x1 + x2 + x3 + x4,
            data = wafer, family = Gamma(link = "log"))
anova(fit1, fit2, test = "F")
summary(fit1)
fit0 <- glm(y ~ x1 + x2 + x3, data = wafer, family = Gamma(link = "log"))
coef(summary(fit0))

The sequential table is not how we choose the model. The last three-factor term has a sequential p-value of 0.69, but that number is only the test of that one interaction with every earlier term retained. We do not want to keep one three-factor interaction and delete its neighbours. We want all four of them in, or all four out.

The three-factor model has residual deviance 0.0084 on 1 degree of freedom. The two-factor model has residual deviance 0.0339 on 5 degrees of freedom. The difference is 0.0255 on 4 and 1 degrees of freedom: F=0.76,\qquad p=0.68. The three-factor terms can go. Comparing main effects with the two-factor model gives F=2.23 on 6 and 5 degrees of freedom, p=0.20. The pairwise products can go as well. Among the four main effects, x_4 has t=-1.10. Dropping it leaves

(Intercept)  5.4155
x1+          0.1211
x2+         -0.2998
x3+          0.1824

so the fitted mean, with each x coded +1 at its high level, is \hat\mu=\exp(5.4155+0.1211\,x_1-0.2998\,x_2+0.1824\,x_3). A quoted value of the x_2 coefficient is -0.2981. The coefficient table, which is the fit, says -0.2998. We use the table.

We go backwards on purpose. Forward selection can drop x_4 at the first step because its main effect is small, and then never see that x_4 changes the effect of the others. Starting from the largest model you can fit, and deleting groups of terms, is the protection against that.

Analysis of deviance

Used in

See also