Three components of a GLM
Section of Introduction to generalised linear models, part of Generalised linear models
Three components of a GLM
| Type | Section |
| Taught in | Introduction to generalised linear models |
| Part | Generalised linear models |
| Questions using it | 1 |
Definition 1 Observations Y_1,\ldots,Y_n follow a generalised linear model when all three of the following are specified.
- Random component. The Y_i are independent, and each belongs to the same exponential family. The functions a, b, c, d and e are common; the parameter \theta_i may depend on i.
- Linear predictor. Covariate vectors {\boldsymbol{X}}_1,\ldots,{\boldsymbol{X}}_n and a coefficient vector \boldsymbol{\beta}=(\beta_0,\ldots,\beta_d)^\top form \eta_i={\boldsymbol{X}}_i^\top\boldsymbol{\beta}.
- Link. A monotone function g connects the mean \mu_i=\mathrm{E}(Y_i) to that predictor, g(\mu_i)={\boldsymbol{X}}_i^\top\boldsymbol{\beta}.
The three choices are separate, and all three are yours. You choose the distribution. You choose which covariates go into {\boldsymbol{X}}_i, and whether to transform them first: log dose is still just a column of {\color{blue}\boldsymbol{X}}. You choose the link. Nothing forces the link that is “canonical” for the distribution you chose.
Definition 2 The link g is canonical when the linear predictor is the natural parameter of the family: g(\mu_i)=b(\theta_i), in the notation of Equation (6) in Introduction to generalised linear models.
For the families above, the canonical links are
- normal: g(\mu)=\mu, the identity, because the natural parameter is the mean;
- Poisson: g(\mu)=\log\mu, because the natural parameter is \log\lambda and \mu=\lambda;
- binomial, including Bernoulli: g(\mu)=\log\{\mu/(1-\mu)\}, the logit;
- gamma, in the shape-and-rate form with mean \mu: g(\mu)=1/\mu, the inverse link.
If you call glm in R and name the family but not the link, you get the canonical link. That is a software default, not a scientific one. The canonical link for the gamma is the inverse link. The link used most often here for a positive continuous response is still the log link, because a slope is then a multiplicative change in the mean, and that is a sentence one can say to a collaborator. Canonical links are the natural mathematical choice. They are not always the appropriate one. You override the default by stating the link yourself.
The normal linear model is the GLM with identity link and normal random component: \mu_i={\boldsymbol{X}}_i^\top\boldsymbol{\beta} and \mathrm{Var}(Y_i)=\sigma^2. Logistic regression is the GLM with Bernoulli random component and the logit link Equation (5) in Introduction to generalised linear models. Writing those two out is not a new model. It is the definition, specialised.
A GLM is an exponential family, a linear predictor, and a link, chosen independently. The canonical link equates the linear predictor with the natural parameter. It is the default in R. It is not a requirement.
Used in
- Taught in Introduction to generalised linear models
- Also used in Underdispersion, positive responses, and quasi-likelihood · Categorical responses · Paired comparisons, and how far a fit is from the data · Smooth means · Logistic regression
- Questions that use it (1) — Q40 (answer)
- All 9 questions on Introduction to generalised linear models
See also
Link function · Exponential family · Mean and variance of an exponential family · Generalised linear model · Canonical link · What a regression statement is hiding · Three examples the linear model does not fit · Runs in the past week