Q29. Comparing Exponential and Gamma lifetime models
[20 marks]
Researchers observed lifetimes {\boldsymbol{x}}=(x_1,\ldots,x_n), with each x_i>0, for a sample of n medical devices. Two models are proposed, with conditionally iid observations under each model: M_1:\quad X_i\mid\lambda\sim\operatorname{Exp}(\lambda), \qquad M_2:\quad X_i\mid\theta\sim\operatorname{Gamma}(3,\theta). Prior experiments suggest the parameter priors \lambda\mid M_1\sim\operatorname{Gamma}(1,3), \qquad \theta\mid M_2\sim\operatorname{Gamma}(1,1). Parameterisation: all Gamma distributions use shape–rate notation, and the exponential parameter is its rate.
(i) [6 marks] Find p({\boldsymbol{x}}\mid M_1), the marginal likelihood of the data under model M_1.
(ii) [6 marks] Find p({\boldsymbol{x}}\mid M_2), the marginal likelihood of the data under model M_2.
(iii) [4 marks] Obtain an expression for BF_{12}, the Bayes factor comparing M_1 with M_2. For the observed data, BF_{12}=1.347. Briefly interpret this number. Based on this evidence alone, which model is preferred?
(iv) [4 marks] Suppose the researchers’ prior belief is that M_2 is three times as likely as M_1. What are the posterior model odds? Which model is now preferred?
Definitions. Write S=\sum_i x_i and Q=\prod_i x_i^2. \operatorname{Exp}(\lambda) uses rate \lambda; \operatorname{Gamma}(a,b) uses shape a and rate b. For model M_j, the marginal likelihood is m_j({\boldsymbol{x}})=\int L_j(\vartheta;{\boldsymbol{x}})p_j(\vartheta)\,d\vartheta and BF_{12}=m_1/m_2. Use the same positive, conditionally independent lifetime sample and the stated proper priors in both models.
A marginal likelihood averages the fully normalised likelihood over the parameter prior. Unlike posterior kernel matching, this calculation must retain all multiplicative constants, including factors involving only the data.
Both calculations use the Gamma integral \int_0^\infty t^{A-1}e^{-Bt}\,dt =\frac{\Gamma(A)}{B^A},\qquad A,B>0.
(i) Marginal likelihood under M_1. For an exponential distribution with rate \lambda, p(x_i\mid\lambda,M_1)=\lambda e^{-\lambda x_i},\qquad x_i>0. Therefore the likelihood and normalised prior are p({\boldsymbol{x}}\mid\lambda,M_1)=\lambda^n e^{-\lambda S}, \qquad p(\lambda\mid M_1)=3e^{-3\lambda},\quad\lambda>0. Integrating their product, \begin{aligned} p({\boldsymbol{x}}\mid M_1) &=\int_0^\infty p({\boldsymbol{x}}\mid\lambda,M_1) p(\lambda\mid M_1)\,d\lambda\\ &=3\int_0^\infty\lambda^n e^{-(S+3)\lambda}\,d\lambda\\ &=\boxed{\frac{3\Gamma(n+1)}{(S+3)^{n+1}}.} \end{aligned} The power \lambda^n corresponds to Gamma shape n+1, not n. The leading 3 comes from the normalised prior and cannot be omitted.
(ii) Marginal likelihood under M_2. For a Gamma distribution with shape 3 and rate \theta, p(x_i\mid\theta,M_2) =\frac{\theta^3}{\Gamma(3)}x_i^2e^{-\theta x_i} =\frac{\theta^3}{2}x_i^2e^{-\theta x_i},\qquad x_i>0. Thus p({\boldsymbol{x}}\mid\theta,M_2)=\frac{Q}{2^n}\theta^{3n}e^{-\theta S}, \qquad p(\theta\mid M_2)=e^{-\theta},\quad\theta>0. Integrating gives \begin{aligned} p({\boldsymbol{x}}\mid M_2) &=\int_0^\infty p({\boldsymbol{x}}\mid\theta,M_2) p(\theta\mid M_2)\,d\theta\\ &=\frac{Q}{2^n}\int_0^\infty\theta^{3n}e^{-(S+1)\theta}\,d\theta\\ &=\boxed{\frac{Q\,\Gamma(3n+1)}{2^n(S+1)^{3n+1}}.} \end{aligned} Although Q/2^n does not involve \theta, it remains part of the marginal likelihood. Dropping it would give the wrong model comparison.
(iii) Bayes factor and its interpretation. By definition, \begin{aligned} BF_{12} &=\frac{p({\boldsymbol{x}}\mid M_1)}{p({\boldsymbol{x}}\mid M_2)}\\ &=\frac{3\Gamma(n+1)}{(S+3)^{n+1}} \frac{2^n(S+1)^{3n+1}}{Q\,\Gamma(3n+1)}. \end{aligned} Hence \boxed{BF_{12} =\frac{3\cdot2^n\Gamma(n+1)(S+1)^{3n+1}} {\Gamma(3n+1)(S+3)^{n+1}\prod_{i=1}^{n}x_i^2}.} The observed value BF_{12}=1.347 means that the marginal likelihood of the data under M_1 is 1.347 times that under M_2, using the specified parameter priors. The data therefore favour M_1, but only slightly. This is not a posterior probability of either model.
Based on the marginal evidence alone, M_1 is preferred. If the prior model probabilities were equal, its posterior probability would be \mathbb{P}(M_1\mid{\boldsymbol{x}})=\frac{1.347}{1+1.347}\approx0.574. Different prior model odds can reverse that posterior preference.
(iv) Unequal prior model probabilities. The prior belief in the question gives \frac{\mathbb{P}(M_1)}{\mathbb{P}(M_2)}=\frac13. Posterior odds equal the Bayes factor times prior odds, so \boxed{ \frac{\mathbb{P}(M_1\mid{\boldsymbol{x}})}{\mathbb{P}(M_2\mid{\boldsymbol{x}})} =1.347\left(\frac13\right)=0.449. } Thus the posterior odds are 0.449:1 in favour of M_1 relative to M_2, or approximately 2.227:1 in favour of M_2. With these two models as the alternatives, \mathbb{P}(M_1\mid{\boldsymbol{x}})=\frac{0.449}{1+0.449}\approx0.310, \mathbb{P}(M_2\mid{\boldsymbol{x}})=\frac1{1+0.449}\approx0.690. Model M_2 is now preferred. The Bayes factor moves the odds toward M_1, but the evidence is too weak to overcome the prior three-to-one preference for M_2.
Key result. Keep the normalising constants when integrating likelihoods. A Bayes factor updates model odds; it is not itself a posterior model probability.