Random component
Topic of Introduction to generalised linear models, part of Generalised linear models
Random component
| Type | Topic |
| Taught in | Introduction to generalised linear models |
| Part | Generalised linear models |
| Questions using it | 10 |
The distribution assumed for the response given the covariates in a GLM, a member of an exponential family such as the normal, Poisson, binomial or gamma. It fixes the mean–variance relationship, \operatorname{Var}(Y)=\phi V(\mu).
In Introduction to generalised linear models
Definition 1 Observations Y_1,\ldots,Y_n follow a generalised linear model when all three of the following are specified.
- Random component. The Y_i are independent, and each belongs to the same exponential family. The functions a, b, c, d and e are common; the parameter \theta_i may depend on i.
- Linear predictor. Covariate vectors {\boldsymbol{X}}_1,\ldots,{\boldsymbol{X}}_n and a coefficient vector \boldsymbol{\beta}=(\beta_0,\ldots,\beta_d)^\top form \eta_i={\boldsymbol{X}}_i^\top\boldsymbol{\beta}.
- Link. A monotone function g connects the mean \mu_i=\mathrm{E}(Y_i) to that predictor, g(\mu_i)={\boldsymbol{X}}_i^\top\boldsymbol{\beta}.
The three choices are separate, and all three are yours. You choose the distribution. You choose which covariates go into {\boldsymbol{X}}_i, and whether to transform them first: log dose is still just a column of {\color{blue}\boldsymbol{X}}. You choose the link. Nothing forces the link that is “canonical” for the distribution you chose.
Definition 2 The link g is canonical when the linear predictor is the natural parameter of the family: g(\mu_i)=b(\theta_i), in the notation of Equation (6) in Introduction to generalised linear models.
For the families above, the canonical links are
- normal: g(\mu)=\mu, the identity, because the natural parameter is the mean;
- Poisson: g(\mu)=\log\mu, because the natural parameter is \log\lambda and \mu=\lambda;
- binomial, including Bernoulli: g(\mu)=\log\{\mu/(1-\mu)\}, the logit;
- gamma, in the shape-and-rate form with mean \mu: g(\mu)=1/\mu, the inverse link.
If you call glm in R and name the family but not the link, you get the canonical link. That is a software default, not a scientific one. The canonical link for the gamma is the inverse link. The link used most often here for a positive continuous response is still the log link, because a slope is then a multiplicative change in the mean, and that is a sentence one can say to a collaborator. Canonical links are the natural mathematical choice. They are not always the appropriate one. You override the default by stating the link yourself.
The normal linear model is the GLM with identity link and normal random component: \mu_i={\boldsymbol{X}}_i^\top\boldsymbol{\beta} and \mathrm{Var}(Y_i)=\sigma^2. Logistic regression is the GLM with Bernoulli random component and the logit link Equation (5) in Introduction to generalised linear models. Writing those two out is not a new model. It is the definition, specialised.
A GLM is an exponential family, a linear predictor, and a link, chosen independently. The canonical link equates the linear predictor with the natural parameter. It is the default in R. It is not a requirement.
Used in
- Taught in Introduction to generalised linear models
- Also used in Underdispersion, positive responses, and quasi-likelihood
- Questions that use it (10) — Q30 (answer) · Q36 (answer) · Q38 (answer) · Q45 (answer) · Q55 · Question 19 (answer) · Question 21 (answer) · C16 (answer) · C27 · C38 (answer)
- All 9 questions on Introduction to generalised linear models
See also
Explanatory variable · Generalised linear model · Exponential family · Gamma regression · Identity link · Poisson regression · Mean and variance of an exponential family · Canonical link