Q54. Poisson regression and overdispersion
[12 marks]
A manufacturing company records the number Y_i of machine failures during one month for machine i. Let x_i denote the age of the machine in years. A statistician proposes the model Y_i\mid x_i\overset{\mathrm{ind}}{\sim}\operatorname{Poisson}(\mu_i), \qquad i=1,\ldots,n, with \log(\mu_i)=\beta_0+\beta_1x_i. Treat the machine ages as fixed covariates.
(a) Write the model explicitly in terms of the conditional mean and conditional variance of Y_i. [2 marks]
(b) Starting from the Poisson probability mass function, \mathbb P(Y_i=y_i\mid x_i) =\frac{\mu_i^{y_i}e^{-\mu_i}}{y_i!}, derive the log-likelihood for \boldsymbol\beta=(\beta_0,\beta_1)^\top, up to an additive constant. Then derive the score vector S(\boldsymbol\beta) =\frac{\partial\ell(\boldsymbol\beta)}{\partial\boldsymbol\beta}. Your final answer should be written in vector form. [4 marks]
(c) Suppose the fitted model gives \hat\beta_1=\log(1.2). Interpret \hat\beta_1 in context. [2 marks]
(d) After fitting the model, the statistician estimates the dispersion parameter as \hat\phi=2.25. What does this suggest about the Poisson variance assumption? [2 marks]
(e) The statistician refits the same mean model using a quasi-Poisson model. State:
1. the new variance assumption;
2. whether the estimate \hat{\boldsymbol\beta} changes;
3. how the standard errors of the regression coefficients are affected.
[2 marks]
Definitions. Write \mathbf x_i=(1,x_i)^\top, \boldsymbol\beta=(\beta_0,\beta_1)^\top, and \mu_i=\exp(\mathbf x_i^\top\boldsymbol\beta). Poisson equidispersion means \operatorname{Var}(Y_i\mid x_i)=\mu_i; quasi-Poisson instead specifies \phi\mu_i. The comparisons use independent observations, fixed ages, the same unpenalised mean model and data, and a common dispersion \phi>0.
(a) Conditional mean and variance. The observations are independent conditional on the fixed machine ages, and the Poisson model gives \boxed{ \mathbb{E}[Y_i\mid x_i]=\mu_i=\exp(\beta_0+\beta_1x_i),\qquad \operatorname{Var}(Y_i\mid x_i)=\mu_i=\exp(\beta_0+\beta_1x_i). } Equality of the conditional mean and variance is the Poisson equidispersion assumption.
(b) Log-likelihood and score vector. For observed nonnegative integer counts y_i, conditional independence gives L(\boldsymbol\beta) =\prod_{i=1}^n\frac{\mu_i^{y_i}e^{-\mu_i}}{y_i!}. Taking logarithms and substituting the log link, \begin{aligned} \ell(\boldsymbol\beta) &=\sum_{i=1}^n\{y_i\log\mu_i-\mu_i-\log(y_i!)\}\\ &=\sum_{i=1}^n\left[ y_i\mathbf x_i^\top\boldsymbol\beta -\exp(\mathbf x_i^\top\boldsymbol\beta)\right]+C, \end{aligned} where C=-\sum_{i=1}^n\log(y_i!) depends on the data but not on \boldsymbol\beta. It can therefore be ignored when maximising or differentiating with respect to \boldsymbol\beta.
The derivative of \mathbf x_i^\top\boldsymbol\beta is \mathbf x_i, and the chain rule gives \frac{\partial}{\partial\boldsymbol\beta} \exp(\mathbf x_i^\top\boldsymbol\beta) =\exp(\mathbf x_i^\top\boldsymbol\beta)\mathbf x_i =\mu_i\mathbf x_i. Hence \boxed{ S(\boldsymbol\beta) =\sum_{i=1}^n\mathbf x_i(y_i-\mu_i) =\begin{pmatrix} \sum_{i=1}^n(y_i-\mu_i)\\[3pt] \sum_{i=1}^n x_i(y_i-\mu_i) \end{pmatrix} =X^\top(\mathbf y-\boldsymbol\mu). } Here X is the n\times2 design matrix with row i equal to \mathbf x_i^\top, while \mathbf y=(y_1,\ldots,y_n)^\top and \boldsymbol\mu=(\mu_1,\ldots,\mu_n)^\top. Thus the score is a 2\times1 vector.
As a check, under the assumed model, \mathbb{E}[S(\boldsymbol\beta)\mid X] =X^\top\{\mathbb{E}[\mathbf Y\mid X]-\boldsymbol\mu\} =\mathbf0.
(c) Interpretation of the age coefficient. For two machine ages differing by one year, the ratio of fitted means is \frac{\hat\mu(x+1)}{\hat\mu(x)} =\frac{\exp\{\hat\beta_0+\hat\beta_1(x+1)\}} {\exp(\hat\beta_0+\hat\beta_1x)} =e^{\hat\beta_1}=1.2. Each additional year of machine age is associated with a 20% higher fitted mean number of failures per month. This is a multiplicative effect on the expected count, not an increase of 0.2 failures or a statement that every machine’s realised count rises by 20%.
(d) Estimated dispersion. An ordinary Poisson model imposes \phi=1. Since \hat\phi=2.25>1, the estimate suggests overdispersion: conditional variability is greater than the Poisson mean–variance relationship allows. Under the fitted dispersion model, the variance is estimated as 2.25\hat\mu_i, compared with the Poisson value \hat\mu_i. This raises concern about Poisson-based standard errors; it does not by itself establish that the log-linear mean model is correct.
(e) Quasi-Poisson refit.
1. Variance assumption. Keep \mathbb{E}[Y_i\mid x_i]=\mu_i=\exp(\beta_0+\beta_1x_i), but use \boxed{\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i,\qquad\phi>0.} Quasi-Poisson specifies this mean–variance relationship rather than a full Poisson probability mass function.
2. Coefficient estimates. For the same data and unpenalised mean model with the same log link, the estimates are unchanged. With a common positive dispersion parameter, the quasi-score is U_Q(\boldsymbol\beta) =\sum_{i=1}^n\frac{y_i-\mu_i}{\phi\mu_i} \frac{\partial\mu_i}{\partial\boldsymbol\beta} =\frac1\phi\sum_{i=1}^n\mathbf x_i(y_i-\mu_i) =\frac1\phi S(\boldsymbol\beta). Thus U_Q=\mathbf0 and S=\mathbf0 have the same solutions; the fitted coefficients and fitted means agree, assuming the usual finite, identifiable fit.
3. Standard errors. Let \widehat W=\operatorname{diag}(\hat\mu_1,\ldots,\hat\mu_n). The usual estimated coefficient covariance matrices are \widehat{\operatorname{Cov}}_P(\hat{\boldsymbol\beta}) =(X^\top\widehat W X)^{-1},\qquad \widehat{\operatorname{Cov}}_Q(\hat{\boldsymbol\beta}) =\hat\phi(X^\top\widehat W X)^{-1}. Taking square roots of their diagonal entries gives \boxed{ \operatorname{SE}_Q(\hat\beta_j) =\sqrt{\hat\phi}\,\operatorname{SE}_P(\hat\beta_j) =1.5\,\operatorname{SE}_P(\hat\beta_j),\quad j=0,1. } Thus the estimated variances are multiplied by 2.25, whereas the standard errors increase by 50%.
Key result. Same log-linear mean model and coefficient estimates; variance \phi\mu_i instead of \mu_i; standard errors multiplied by \sqrt{\hat\phi}, not by \hat\phi.