Probit link
Topic of Introduction to generalised linear models, part of Generalised linear models · Also in Logistic regression
Probit link
| Type | Topic |
| Taught in | Introduction to generalised linear models · Logistic regression |
| Part | Generalised linear models · Statistical learning |
| Questions using it | 3 |
g(p)=\Phi^{-1}(p), the standard normal quantile function. It fits binary data much like the logit, with coefficients smaller by a factor of about 1.6 to 1.8, but it has no odds-ratio reading.
In Introduction to generalised linear models
The probit link replaces the logistic curve by a standard normal distribution function. If \Phi is the cumulative distribution function of \mathrm{N}(0,1), \Phi^{-1}(p_i)={\boldsymbol{X}}_i^\top\boldsymbol{\beta} \qquad\text{equivalently}\qquad p_i=\Phi({\boldsymbol{X}}_i^\top\boldsymbol{\beta}). Read it as a quantile. Compute {\boldsymbol{X}}_i^\top\boldsymbol{\beta}, find that point on the standard normal axis, and take the probability to its left. That left-tail probability is p_i. Any other continuous distribution function would give a link of the same type. The normal is the default.
In Logistic regression
From Logistic regression
Both links are smooth curves from 0 to 1, so the choice is not forced by the range of a probability. The scale of the linear predictor can be absorbed into \beta, which means the two curves should be compared after they are given the same slope at 0.
Differentiate \sigma(\eta)=(1+e^{-\eta})^{-1}. The chain rule gives
\sigma'(\eta) =(1+e^{-\eta})^{-2}\,e^{-\eta} =\frac{1}{1+e^{-\eta}}\cdot\frac{e^{-\eta}}{1+e^{-\eta}} =\sigma(\eta)\bigl(1-\sigma(\eta)\bigr),
because 1-\sigma(\eta)=e^{-\eta}/(1+e^{-\eta}). Hence \sigma'(0)=\sigma(0)(1-\sigma(0))=(1/2)(1/2)=1/4. The same identity gives the symmetry \sigma(-\eta)=1-\sigma(\eta), since
\sigma(-\eta)=\frac{1}{1+e^{\eta}}=\frac{e^{-\eta}}{e^{-\eta}+1}=1-\sigma(\eta).
The rescaled normal distribution function \Phi(c\eta) has slope c\,\varphi(0) at 0, where \varphi(0)=1/\sqrt{2\pi} is the standard normal density. Equating the slopes,
\frac{c}{\sqrt{2\pi}}=\frac14, \qquad c=\frac{\sqrt{2\pi}}{4}=\sqrt{\frac{\pi}{8}},
the last step because (\sqrt{2\pi}/4)^2=2\pi/16=\pi/8. With that scaling the curves nearly agree. The logistic function approaches 0 and 1 a little more slowly, so it is the heavier-tailed of the two links. The later algebra uses the logistic derivative \sigma(1-\sigma), which is why the logistic link is the one carried forward.
Used in
- Taught in Introduction to generalised linear models · Logistic regression
- Also used in Paired comparisons, and how far a fit is from the data · Logistic regression
- Questions that use it (3) — Q38 (answer) · Question 21 · C20
- All 9 questions on Introduction to generalised linear models
See also
Logit link · Complementary log-log link · Identity link · Log link · Inverse link · Canonical link · Exponential family · Mean and variance of an exponential family · Generalised linear model · What a regression statement is hiding