Question 15. Battery lifetimes and a Bayes factor

Topic: Bayesian hypotheses and model comparison · more questions on this topic
‹ Question 8 Question 18 ›

Researchers are attempting to determine the distribution of lifetimes of a type of battery. They have measured the lifetimes (years) of n of these batteries, recorded as x_1,\ldots,x_n, which we assume were the result of a random sample and we will assume that the observations could be considered independent and identically distributed. Two models were proposed:

M_1\colon x\sim\operatorname{Exp}(\theta) \qquad\text{and}\qquad M_2\colon x\sim\operatorname{Weibull}(5,\theta).

This Weibull density is given by

f(x)=5\theta x^{4}e^{-\theta x^{5}}, \qquad\text{for }x>0.

Consider the prior distribution \theta\sim\operatorname{Exp}(1).

(a) Find the marginal likelihood under model M_1. [3 marks]

(b) Find the marginal likelihood under model M_2. [3 marks]

(c) Obtain an expression for the Bayes factor comparing M_1 to M_2. [2 marks]

Worked answer (click to show)

Definitions. Under M_1, \theta is an exponential rate. Under M_2, its meaning is fixed by the supplied Weibull density 5\theta x^4e^{-\theta x^5}, rather than by a software scale convention. In both models the prior density is e^{-\theta} on \theta>0. Write S_1=\sum_i x_i, S_5=\sum_i x_i^5 and P_4=\prod_i x_i^4. The marginal likelihood averages a sampling density over the specified parameter prior.

(a) Let S_1=\sum_{i=1}^{n}x_i. Independence gives, for \theta>0,

p({\boldsymbol{x}}\mid\theta,M_1) = \prod_{i=1}^{n}\theta e^{-\theta x_i} = \theta^{n}e^{-\theta S_1}.

Since S_1+1>0,

p({\boldsymbol{x}}\mid M_1) = \int_0^{\infty}\theta^{n}e^{-\theta(S_1+1)}\,d\theta.

Put u=(S_1+1)\theta. Then d\theta=du/(S_1+1) and u runs over (0,\infty), so

\begin{aligned} p({\boldsymbol{x}}\mid M_1) &= (S_1+1)^{-(n+1)}\int_0^{\infty}u^{n}e^{-u}\,du = \frac{\Gamma(n+1)}{(S_1+1)^{n+1}}. \end{aligned}

Here n is a positive integer, so \Gamma(n+1)=n!.

\boxed{p({\boldsymbol{x}}\mid M_1) = \dfrac{n!}{\bigl(1+\sum_{i=1}^{n}x_i\bigr)^{n+1}}.}

(b) Let P_4=\prod_{i=1}^{n}x_i^{4} and S_5=\sum_{i=1}^{n}x_i^{5}. The given density and independence give

p({\boldsymbol{x}}\mid\theta,M_2) = \prod_{i=1}^{n}5\theta x_i^{4}e^{-\theta x_i^{5}} = 5^{n}\theta^{n}P_4 e^{-\theta S_5}, \qquad\theta>0.

Thus

p({\boldsymbol{x}}\mid M_2) = 5^{n}P_4\int_0^{\infty}\theta^{n}e^{-(S_5+1)\theta}\,d\theta.

Now S_5+1>0. The same substitution as in (a) gives \int_0^{\infty}\theta^{n}e^{-(S_5+1)\theta}\,d\theta=n!/(S_5+1)^{n+1}. The factor 5^{n}P_4 does not depend on \theta, but it belongs to the normalised sampling density, so it remains in the marginal likelihood.

\boxed{p({\boldsymbol{x}}\mid M_2) = \dfrac{5^{n}\,n!\prod_{i=1}^{n}x_i^{4}}{\bigl(1+\sum_{i=1}^{n}x_i^{5}\bigr)^{n+1}}.}

(c) By (a) and (b),

\begin{aligned} \operatorname{BF}_{12} &= \frac{p({\boldsymbol{x}}\mid M_1)}{p({\boldsymbol{x}}\mid M_2)} = \frac{n!/(1+S_1)^{n+1}}{5^{n}n!\,P_4/(1+S_5)^{n+1}} = \frac{1}{5^{n}P_4} \Biggl(\frac{1+S_5}{1+S_1}\Biggr)^{n+1}. \end{aligned}

\boxed{\operatorname{BF}_{12} = \dfrac{1}{5^{n}\prod_{i=1}^{n}x_i^{4}} \Biggl( \dfrac{1+\sum_{i=1}^{n}x_i^{5}}{1+\sum_{i=1}^{n}x_i} \Biggr)^{n+1}.}

Thus \operatorname{BF}_{12}>1 favours M_1, and \operatorname{BF}_{12}<1 favours M_2.

The transformation U=X^5 clarifies the second integration: under M_2, U\mid\theta\sim\operatorname{Exp}(\theta). However, the observed data are the x_i, so the transformation Jacobian \prod_i5x_i^4=5^nP_4 remains in their likelihood. Dropping it would compare densities expressed for different observed variables. In posterior inference within M_2 it is constant in \theta and can be omitted; in this model comparison it cannot.

The prior in each model must also be normalised. Using a density proportional to one on the positive line in place of \operatorname{Exp}(1) would introduce an unspecified multiplicative constant into each marginal likelihood. The stated proper priors avoid that ambiguity. Posterior model odds are obtained only after multiplying BF_{12} by the prior model odds.