Overdispersion
Topic of Standard errors, and models for counts, part of Generalised linear models · Also in Random variables and distributions
Overdispersion
| Type | Topic |
| Taught in | Standard errors, and models for counts · Random variables and distributions |
| Part | Generalised linear models · Probability |
| Questions using it | 7 |
Count data are overdispersed when their variance exceeds what the model allows, usually a Poisson variance equal to the mean. Ignoring it makes standard errors too small. A quasi-Poisson dispersion or a negative binomial model allows for it.
In Standard errors, and models for counts
Unless there is a strong reason to insist on a pure Poisson process, use
\mathrm{Var}(Y_i\mid {\boldsymbol{X}}_i)=\phi\mu_i=\phi\exp({\boldsymbol{X}}_i^\top\boldsymbol{\beta}).
Here \phi>1 is overdispersion and \phi<1 is underdispersion. The case \phi=1 is the Poisson. This is a quasi-Poisson model. The variance function is a legitimate description of spread. Inside the exponential family, a variance exactly proportional to the mean forces the Poisson, so there is no exponential-family distribution with a free \phi in Equation (4) in Standard errors, and models for counts. You cannot simulate a new count from a quasi-Poisson specification alone. We will use it as a description. A stricter view, which reserves the name model for something one can simulate from, would refuse the name. We will come back to that distinction when we treat quasi-likelihood properly. For fitting, glm only needs the mean and the variance, so family = quasipoisson runs.
In Random variables and distributions
The generating function is G(z)=\sum_x e^{-\lambda}(\lambda z)^x/x!=e^{\lambda(z-1)}. Then G'(1)=\lambda and G''(1)=\lambda^2, so the mean is \lambda and the variance is \lambda^2+\lambda-\lambda^2=\lambda. Equal mean and variance is the signature of a Poisson count, and data whose variance is well above the mean are called overdispersed; see Standard errors, and models for counts.
Used in
- Taught in Standard errors, and models for counts · Random variables and distributions
- Also used in Random variables and distributions
- Questions that use it (7) — Q33 (answer) · Q34 (answer) · Q47 (answer) · Q54 (answer) · Question 19 (answer) · C22 · C28
- All 7 questions on Standard errors, and models for counts
See also
Underdispersion · Quasi-Poisson · Negative binomial distribution · Conway–Maxwell–Poisson distribution · Count responses · Variance · Standard error · Dispersion parameter