Canonical link
Definition taught in Introduction to generalised linear models, part of Generalised linear models
Canonical link
| Type | Definition |
| Taught in | Introduction to generalised linear models |
| Part | Generalised linear models |
| Worked examples | 2 |
| Questions using it | 2 |
The link g is canonical when the linear predictor is the natural parameter of the family: g(\mu_i)=b(\theta_i), in the notation of Equation (6) in Introduction to generalised linear models.
Examples
Q57. Negative-binomial canonical link
Question page: Q57. Negative-binomial canonical link
[6 marks]
Let Y have a negative-binomial distribution with probability mass function f(y;\theta) =\binom{y+r-1}{r-1}\theta^r(1-\theta)^y, \qquad y=0,1,2,\ldots, where r>0 is known and 0<\theta<1.
(a) Show that this distribution belongs to the linear exponential family by expressing its probability mass function in the form f(y;\theta) =\exp\left\{ \frac{y\,b(\theta)+c(\theta)}{e(\phi)}+d(y;\phi) \right\}. Identify the natural parameter b(\theta). [3 marks]
(b) Given that \mu=\mathbb{E}(Y)=\frac{r(1-\theta)}{\theta}, derive the canonical link function g(\mu). [3 marks]
Definitions. Here Y counts failures, with support \{0,1,2,\ldots\}, known r>0, and 0<\theta<1. For non-integer r, interpret the binomial coefficient as \binom{y+r-1}{r-1}=\Gamma(y+r)/\{\Gamma(r)\Gamma(y+1)\}. In the stated linear exponential-family form, the natural parameter is b(\theta); the canonical link is g(\mu)=b\{\theta(\mu)\}. We use scale convention e(\phi)=1.
(a) Exponential-family representation. The support \{0,1,2,\ldots\} does not depend on \theta. Take logarithms of the supplied mass function: \begin{aligned} \log f(y;\theta) &=\log\left\{\binom{y+r-1}{r-1}\theta^r(1-\theta)^y\right\}\\ &=y\log(1-\theta)+r\log\theta +\log\binom{y+r-1}{r-1}. \end{aligned} Since r is known, there is no free dispersion parameter to estimate here. Choose e(\phi)=1 and identify \boxed{ b(\theta)=\log(1-\theta),\qquad c(\theta)=r\log\theta,\qquad e(\phi)=1, } \boxed{d(y;\phi)=\log\binom{y+r-1}{r-1}.} The symbol \phi is redundant in this choice: d depends on the known r, not on the unknown \theta. Substitution gives f(y;\theta) =\exp\left\{y\log(1-\theta)+r\log\theta+ \log\binom{y+r-1}{r-1}\right\}, exactly the required form. In particular, \boxed{\eta=b(\theta)=\log(1-\theta)<0.} The natural parameter is \log(1-\theta), not \log\theta: it is the coefficient multiplying the observation y.
The Gamma-function convention in Definitions makes this representation valid for every stated r>0.
(b) Canonical link. Start with the supplied mean and solve for the original parameter: \begin{aligned} \mu&=\frac{r(1-\theta)}{\theta},\\ \mu\theta&=r-r\theta,\\ (\mu+r)\theta&=r,\\ \theta&=\frac r{\mu+r},\qquad 1-\theta=\frac{\mu}{\mu+r}. \end{aligned} Now substitute into the natural parameter from part (a): \boxed{ g(\mu)=b\{\theta(\mu)\} =\log\left(\frac{\mu}{\mu+r}\right) =\log\mu-\log(\mu+r),\qquad\mu>0. } Thus a canonical-link regression would set \eta_i=\mathbf X_i^\top\boldsymbol\beta=g(\mu_i), with \eta_i<0 to keep the means valid. Solving for the mean gives e^\eta=\frac{\mu}{\mu+r} \quad\Longrightarrow\quad \mu(1-e^\eta)=re^\eta \quad\Longrightarrow\quad \boxed{\mu=\frac{re^\eta}{1-e^\eta},\qquad\eta<0.}
Independent check from the normalising term. Write the log mass function as \log f(y;\eta)=y\eta-A(\eta)+d(y), \qquad A(\eta)=-r\log(1-e^\eta). Differentiating the identity \sum_y f(y;\eta)=1 gives 0=\sum_y\{y-A'(\eta)\}f(y;\eta), so \mathbb{E}(Y)=A'(\eta). Here A'(\eta)=\frac{re^\eta}{1-e^\eta}, which is exactly the inverse link just derived.
A log link can be chosen for a negative-binomial regression, but it is not the canonical link derived here.
C17. Links
Question page: C17. Links
What are a link function, an inverse link and a canonical link?
Definitions/notation. The link g maps the conditional mean to the linear predictor, g(\mu_i)=\eta_i, and its inverse gives \mu_i=g^{-1}(\eta_i). A canonical link equates \eta_i with the natural parameter in the chosen linear exponential-family representation; it is a transformation of the mean, not of each observed response.
Used in
- Taught in Introduction to generalised linear models, where the derivation and examples are.
- Also used in Underdispersion, positive responses, and quasi-likelihood · Smooth means
- Questions that use it (2) — Q57 · C17
- All 9 questions on Introduction to generalised linear models
See also
Exponential family · Mean and variance of an exponential family · Generalised linear model