Q56. Exponential regression and the score
[5 marks]
Researchers study how long electronic components last. Let Y_i>0 be the lifetime of component i, and x_i its operating temperature. Assume independent responses with density f(y_i\mid\mu_i) =\frac1{\mu_i}\exp\left(-\frac{y_i}{\mu_i}\right), \qquad y_i>0, and a log link for the mean: \log\mu_i=\beta_0+\beta_1x_i =\mathbf X_i^\top\boldsymbol\beta,\qquad \mathbf X_i=(1,x_i)^\top,\qquad i=1,\ldots,n. Treat the operating temperatures as fixed. Here \mu_i=\mathbb{E}[Y_i\mid x_i]; the exponential rate is 1/\mu_i.
(a) Starting from the density, form the likelihood and show that \ell(\boldsymbol\beta) =-\sum_{i=1}^n\left[ \mathbf X_i^\top\boldsymbol\beta +Y_i e^{-\mathbf X_i^\top\boldsymbol\beta}\right]. [2 marks]
(b) Derive the score vector \mathbf S(\boldsymbol\beta) =\frac{\partial\ell}{\partial\boldsymbol\beta}. Express your answer using Y_i,\mu_i,\mathbf X_i. [2 marks]
(c) Show that \mathbb{E}[\mathbf S(\boldsymbol\beta)]=\mathbf0. [1 mark]
Definitions. The exponential distribution is parameterised by mean \mu_i>0, so its rate is 1/\mu_i, not \mu_i. Let \mathbf X_i=(1,x_i)^\top, \boldsymbol\beta=(\beta_0,\beta_1)^\top, and \eta_i=\mathbf X_i^\top\boldsymbol\beta. The score is \mathbf S=\partial\ell/\partial\boldsymbol\beta, differentiating with the observed lifetimes fixed. Sampling is independent conditional on the fixed covariates.
(a) Likelihood and log-likelihood. Write \eta_i=\mathbf X_i^\top\boldsymbol\beta. The log link implies \mu_i=e^{\eta_i},\qquad\frac1{\mu_i}=e^{-\eta_i}, \qquad\log\mu_i=\eta_i. Independence gives, for the observed positive lifetimes, L(\boldsymbol\beta) =\prod_{i=1}^n\frac1{\mu_i}\exp\left(-\frac{Y_i}{\mu_i}\right) =\exp\left[-\sum_{i=1}^n\left(\log\mu_i+\frac{Y_i}{\mu_i}\right)\right]. The observed Y_i are held fixed when differentiating the likelihood. Taking logarithms and substituting the log link gives \boxed{ \ell(\boldsymbol\beta) =-\sum_{i=1}^n\left(\log\mu_i+\frac{Y_i}{\mu_i}\right) =-\sum_{i=1}^n\left[ \mathbf X_i^\top\boldsymbol\beta +Y_i e^{-\mathbf X_i^\top\boldsymbol\beta}\right]. } This is the exact log-likelihood for positive observations; the support does not depend on \boldsymbol\beta.
(b) Score vector. Since \partial\eta_i/\partial\boldsymbol\beta=\mathbf X_i, the chain rule gives \frac{\partial}{\partial\boldsymbol\beta} \left[-\eta_i-Y_i e^{-\eta_i}\right] =-\mathbf X_i+Y_i e^{-\eta_i}\mathbf X_i. The second term is positive because differentiating e^{-\eta_i} introduces a second minus sign. Summing and using e^{-\eta_i}=1/\mu_i, \boxed{ \mathbf S(\boldsymbol\beta) =\sum_{i=1}^n\mathbf X_i\left(\frac{Y_i}{\mu_i}-1\right) =\begin{pmatrix} \sum_{i=1}^n(Y_i/\mu_i-1)\\[3pt] \sum_{i=1}^n x_i(Y_i/\mu_i-1) \end{pmatrix}. } This is a 2\times1 vector. Unlike the Poisson log-link score, it uses the relative residual Y_i/\mu_i-1=(Y_i-\mu_i)/\mu_i.
(c) Expected score. Let X denote the fixed design matrix whose ith row is \mathbf X_i^\top. Under the assumed model, \mathbb{E}[Y_i\mid X]=\mu_i. For fixed \boldsymbol\beta, both \mathbf X_i and \mu_i are nonrandom, so linearity of expectation gives \begin{aligned} \mathbb{E}[\mathbf S(\boldsymbol\beta)\mid X] &=\sum_{i=1}^n\mathbf X_i \left(\frac{\mathbb{E}[Y_i\mid X]}{\mu_i}-1\right)\\ &=\sum_{i=1}^n\mathbf X_i \left(\frac{\mu_i}{\mu_i}-1\right) =\boxed{\mathbf0}. \end{aligned} Since the covariates are fixed, this is the requested \mathbb{E}[\mathbf S(\boldsymbol\beta)]=\mathbf0: expectation is over repeated responses generated at the same parameter \boldsymbol\beta.