Q48. Conjugate Bayesian inference for a Gamma rate
[10 marks]
Let X_1,\ldots,X_n be conditionally independent given \lambda, with density f(x_i\mid\lambda) =\frac{\lambda^\alpha}{\Gamma(\alpha)} x_i^{\alpha-1}e^{-\lambda x_i},\qquad x_i>0, where \alpha>0 is known and \lambda>0 is unknown.
Suppose \lambda\sim\operatorname{Gamma}(a,b), where a,b>0, with prior density p(\lambda)=\frac{b^a}{\Gamma(a)}\lambda^{a-1}e^{-b\lambda}.
(a) Derive the posterior distribution of \lambda\mid{\boldsymbol{x}}, identifying its parameters. Show the likelihood and posterior kernels. [4 marks]
(b) State the posterior mean \mathbb{E}[\lambda\mid{\boldsymbol{x}}]. [1 mark]
(c) Derive the maximum likelihood estimator \widehat\lambda_{\mathrm{ML}}. [2 marks]
(d) Express the posterior mean in the form \mathbb{E}[\lambda\mid{\boldsymbol{x}}] =w_n\mathbb{E}[\lambda]+(1-w_n)\widehat\lambda_{\mathrm{ML}}. Identify w_n, verify that this is a convex combination, and determine the limits of the weights as n\to\infty, assuming \bar x_n\to m>0 and fixed prior parameters. [3 marks]
Definitions. \operatorname{Gamma}(A,B) uses shape A and rate B, with kernel \lambda^{A-1}e^{-B\lambda} on \lambda>0 and mean A/B. The sampling shape \alpha>0 is known, while the rate \lambda is unknown. Write T_n=\sum_i x_i=n\bar x_n>0. The prior parameters a,b>0 are fixed; the asymptotic statement uses the specified \bar x_n\to m>0.
(a) Likelihood and posterior. Conditional independence gives \begin{aligned} L(\lambda;{\boldsymbol{x}}) &=\prod_{i=1}^n \left\{\frac{\lambda^\alpha}{\Gamma(\alpha)} x_i^{\alpha-1}e^{-\lambda x_i}\right\}\\ &=\frac{\prod_{i=1}^n x_i^{\alpha-1}}{\Gamma(\alpha)^n} \lambda^{n\alpha}e^{-\lambda T_n}. \end{aligned} The leading factor does not involve \lambda, so \boxed{L(\lambda;{\boldsymbol{x}})\propto\lambda^{n\alpha}e^{-T_n\lambda}, \qquad\lambda>0.} Multiplying by the prior kernel, \begin{aligned} p(\lambda\mid{\boldsymbol{x}}) &\propto\lambda^{n\alpha}e^{-T_n\lambda} \lambda^{a-1}e^{-b\lambda}\\ &=\lambda^{a+n\alpha-1}e^{-(b+T_n)\lambda}. \end{aligned} Matching \lambda^{A-1}e^{-B\lambda} gives \boxed{\lambda\mid{\boldsymbol{x}}\sim \operatorname{Gamma}(a+n\alpha,\ b+T_n) \quad\text{(shape--rate)},\qquad\lambda>0.} Thus the Gamma family is conjugate for the unknown rate.
(b) Posterior mean. A shape–rate Gamma distribution has mean shape divided by rate: \boxed{\mathbb{E}[\lambda\mid{\boldsymbol{x}}]=\frac{a+n\alpha}{b+T_n}.}
(c) Maximum likelihood estimator. The log-likelihood, up to a constant, is \ell(\lambda)=n\alpha\log\lambda-T_n\lambda+C. Its derivatives are \ell'(\lambda)=\frac{n\alpha}{\lambda}-T_n, \qquad \ell''(\lambda)=-\frac{n\alpha}{\lambda^2}<0. Solving \ell'(\lambda)=0 therefore gives the unique maximiser \boxed{\widehat\lambda_{\mathrm{ML}}=\frac{n\alpha}{T_n} =\frac{\alpha}{\bar x_n}.}
(d) Convex combination and limiting weights. The prior mean is \mathbb{E}[\lambda]=a/b. Split the posterior mean as follows: \begin{aligned} \frac{a+n\alpha}{b+T_n} &=\frac{b}{b+T_n}\frac ab +\frac{T_n}{b+T_n}\frac{n\alpha}{T_n}\\ &=w_n\mathbb{E}[\lambda]+(1-w_n)\widehat\lambda_{\mathrm{ML}}, \end{aligned} where \boxed{w_n=\frac{b}{b+T_n}=\frac{b}{b+n\bar x_n}, \qquad 1-w_n=\frac{T_n}{b+T_n}.} Because b>0 and T_n>0, both weights are strictly between zero and one, and \frac{b}{b+T_n}+\frac{T_n}{b+T_n}=1. Hence this is a convex combination of the prior mean and the MLE. Notice that the weights depend on the observed total T_n, not just on n.
Under \bar x_n\to m>0, we have T_n=n\bar x_n\to\infty. With fixed prior parameters, \boxed{w_n\longrightarrow0,\qquad1-w_n\longrightarrow1.} The prior-mean contribution consequently vanishes and the data receive asymptotically all the weight. In fact, both estimates have the same limit: \widehat\lambda_{\mathrm{ML}}=\frac\alpha{\bar x_n} \longrightarrow\frac\alpha m, \qquad \mathbb{E}[\lambda\mid{\boldsymbol{x}}] =\frac{a/n+\alpha}{b/n+\bar x_n} \longrightarrow\frac\alpha m.
Key result. For this rate parameter, the prior weight is b/(b+\sum_i x_i). Derive the weights from the actual posterior mean; they are not always “prior sample size divided by total sample size.”