Q46. Multinomial logistic regression
[10 marks]
A response has categories 0,1,2, with probabilities p_0(x),p_1(x),p_2(x). Consider \log\frac{p_1(x)}{p_0(x)}=\alpha_1+\beta_1x, \qquad \log\frac{p_2(x)}{p_0(x)}=\alpha_2+\beta_2x.
(a) Derive expressions for all three probabilities and verify that they sum to one. [4 marks]
(b) Interpret e^{\beta_1}. [2 marks]
(c) Derive \log[p_2(x)/p_1(x)]. [2 marks]
(d) If category 1 becomes the reference category, express the new intercepts and slopes in terms of the original parameters. Do the fitted probabilities change? [2 marks]
Definitions. p_0(x),p_1(x),p_2(x) are probabilities of three mutually exclusive, exhaustive categories, summing to one. With category 0 as baseline, define \eta_j(x)=\log\{p_j(x)/p_0(x)\}=\alpha_j+\beta_jx, for j=1,2. These are baseline-category log ratios, not one-versus-rest logits \log\{p_j/(1-p_j)\}. The predictors are finite, so all fitted probabilities are positive.
(a) The three probabilities. Exponentiating the model equations gives p_1(x)=e^{\eta_1(x)}p_0(x),\qquad p_2(x)=e^{\eta_2(x)}p_0(x). Since the probabilities sum to one, p_0(x)\{1+e^{\eta_1(x)}+e^{\eta_2(x)}\}=1. With D(x)=1+e^{\eta_1(x)}+e^{\eta_2(x)}, \boxed{p_0(x)=\frac1{D(x)},\qquad p_1(x)=\frac{e^{\eta_1(x)}}{D(x)},\qquad p_2(x)=\frac{e^{\eta_2(x)}}{D(x)}.} All are positive and their sum is D(x)/D(x)=1.
(b) Interpretation of e^{\beta_1}. For a one-unit increase in x, \boxed{ \frac{p_1(x+1)/p_0(x+1)}{p_1(x)/p_0(x)}=e^{\beta_1}. } Thus e^{\beta_1} is the multiplicative change in the odds of category 1 relative to category 0. It is not generally the multiplier for p_1(x) itself, or for the odds of category 1 against all other categories combined.
(c) Comparing categories 2 and 1. Subtract the two original log-ratio equations: \boxed{ \log\frac{p_2(x)}{p_1(x)} =\log\frac{p_2(x)}{p_0(x)}-\log\frac{p_1(x)}{p_0(x)} =(\alpha_2-\alpha_1)+(\beta_2-\beta_1)x. }
(d) Taking category 1 as the reference. The two new equations are \log\frac{p_0(x)}{p_1(x)}=-\alpha_1-\beta_1x, \qquad \log\frac{p_2(x)}{p_1(x)} =(\alpha_2-\alpha_1)+(\beta_2-\beta_1)x. Thus the new intercept and slope are (-\alpha_1,-\beta_1) for category 0, and (\alpha_2-\alpha_1,\beta_2-\beta_1) for category 2. The fitted probabilities do not change: this is a reparametrisation. For example, p_1(x)=\frac1{1+e^{-\eta_1(x)}+e^{\eta_2(x)-\eta_1(x)}} =\frac{e^{\eta_1(x)}}{D(x)}, which is the original expression, and the other probabilities follow by their corresponding ratios.