Q11. Bayes factors and posterior odds

Topic: Bayesian hypotheses and model comparison · more questions on this topic
‹ An IQ threshold Q21 ›

Suppose X\mid\theta\sim\operatorname{Bin}(n,\theta). Consider M_0:\theta=\frac12, \qquad M_1:\theta\sim\operatorname{Unif}(0,1). Under M_1, the stated distribution is the prior for \theta. Write m_j(x)=p(x\mid M_j).

(a) Derive m_0(x) and m_1(x). Retain the constants in the binomial probability mass function. [4 marks]

(b) Obtain B_{10}=\frac{m_1(x)}{m_0(x)}. Explain how it relates prior model odds to posterior model odds. [3 marks]

(c) Suppose n=3, x=3, and \mathbb{P}(M_1)=1/5. Calculate B_{10} and \mathbb{P}(M_1\mid x). Explain how the data can favour M_1 while M_0 still has the larger posterior probability. [3 marks]

Worked answer (click to show)

Definitions. M_0 fixes \theta=1/2; M_1 assigns \theta\sim\operatorname{Beta}(1,1). Write m_j(x)=\mathbb P(X=x\mid M_j) for the marginal likelihood and B_{10}=m_1(x)/m_0(x) for the Bayes factor favouring M_1. The Beta integral is B(a,b)=\int_0^1u^{a-1}(1-u)^{b-1}\,du. Prior probabilities of the two models are distinct from their priors for \theta.

(a) Marginal likelihoods

Under M_0, the marginal probability of observing X=x is m_0(x) = \mathbb{P}(X=x\mid M_0) = \binom{n}{x}\left(\frac12\right)^x \left(\frac12\right)^{n-x} = \boxed{\binom{n}{x}2^{-n}}.

Under M_1, integrate the likelihood with respect to the uniform prior: \begin{aligned} m_1(x) &= \mathbb{P}(X=x\mid M_1)\\ &= \binom{n}{x} \int_0^1\theta^x(1-\theta)^{n-x}\,d\theta\\ &= \binom{n}{x}B(x+1,n-x+1)\\ &= \binom{n}{x} \frac{x!(n-x)!}{(n+1)!}\\ &= \boxed{\frac{1}{n+1}}. \end{aligned}

(b) Bayes factor

The Bayes factor in favour of M_1 over M_0 is \boxed{ B_{10} = \frac{m_1(x)}{m_0(x)} = \frac{2^n}{(n+1)\binom{n}{x}}. } Values greater than one favour M_1, while values less than one favour M_0.

(c) Posterior model odds

Posterior odds equal the Bayes factor multiplied by prior odds: \boxed{ \frac{\mathbb{P}(M_1\mid x)}{\mathbb{P}(M_0\mid x)} = B_{10} \frac{\mathbb{P}(M_1)}{\mathbb{P}(M_0)}. }

(d) Numerical comparison

For n=3 and x=3, m_0(3)=\frac18, \qquad m_1(3)=\frac14, \qquad B_{10}=2. Thus the observed data are twice as probable under M_1 as under M_0, so the data favour M_1.

If \mathbb{P}(M_1)=\frac15, \qquad \mathbb{P}(M_0)=\frac45, then the prior odds for M_1 are 1/4. Hence \frac{\mathbb{P}(M_1\mid x)}{\mathbb{P}(M_0\mid x)} =2\left(\frac14\right) =\frac12. Therefore \boxed{ \mathbb{P}(M_1\mid x)=\frac13, \qquad \mathbb{P}(M_0\mid x)=\frac23. } The data favour M_1, but the prior favoured M_0 by four to one. The evidence from this small sample is not strong enough to overturn that prior preference, so M_0 remains more probable a posteriori.