Posterior-odds factorisation

Theorem taught in Bayesian hypotheses and model comparison, part of Bayesian statistics

Posterior-odds factorisation

Type Theorem
Taught in Bayesian hypotheses and model comparison
Part Bayesian statistics
Worked examples 2
Questions using it 8

For two models M_1 and M_2, Bayes’ rule gives

\mathbb{P}(M_j\mid x) =\frac{p(x\mid M_j)\mathbb{P}(M_j)}{p(x)}.

Taking the ratio cancels the common evidence p(x): \begin{aligned} \frac{\mathbb{P}(M_1\mid x)}{\mathbb{P}(M_2\mid x)} &=\frac{p(x\mid M_1)\mathbb{P}(M_1)/p(x)} {p(x\mid M_2)\mathbb{P}(M_2)/p(x)}\\ &=\underbrace{\frac{p(x\mid M_1)}{p(x\mid M_2)}}_{ \operatorname{BF}_{12}} \underbrace{\frac{\mathbb{P}(M_1)}{\mathbb{P}(M_2)}}_{ \text{prior odds}}. \end{aligned}

Examples

Q21. Evidence and posterior model probabilities

Question page: Q21. Evidence and posterior model probabilities

[4 marks]

An analysis produces a Bayes factor of 4 in favour of model M_1 over model M_0. Before observing the data, the researcher regarded M_0 as nine times as probable as M_1. The researcher concludes:

“The Bayes factor favours M_1, so M_1 must now have the greater posterior probability.”

Assess this conclusion. Explain how the Bayes factor changes the model odds, and identify which model has the greater posterior probability.

Definitions. The Bayes factor is B_{10}=p({\boldsymbol{x}}\mid M_1)/p({\boldsymbol{x}}\mid M_0). Odds in favour of M_1 are \mathbb P(M_1)/\mathbb P(M_0), with the analogous conditional ratio for posterior odds. The two models exhaust the comparison, so their probabilities sum to one. Here B_{10}=4 and the prior odds are 1:9.

A Bayes factor updates prior model odds according to \frac{\mathbb{P}(M_1\mid{\boldsymbol{x}})}{\mathbb{P}(M_0\mid{\boldsymbol{x}})} =B_{10} \frac{\mathbb{P}(M_1)}{\mathbb{P}(M_0)}. Because M_0 was initially nine times as probable as M_1, the prior odds in favour of M_1 are \frac{\mathbb{P}(M_1)}{\mathbb{P}(M_0)}=\frac19. With B_{10}=4, the posterior odds become \boxed{ \frac{\mathbb{P}(M_1\mid{\boldsymbol{x}})}{\mathbb{P}(M_0\mid{\boldsymbol{x}})} =4\left(\frac19\right)=\frac49. } Therefore, \mathbb{P}(M_1\mid{\boldsymbol{x}})=\frac4{13}, \qquad \mathbb{P}(M_0\mid{\boldsymbol{x}})=\frac9{13}. The data favour M_1 relative to the prior because B_{10}>1, but the evidence is not strong enough to overcome the initial nine-to-one odds in favour of M_0. Thus \boxed{ \mathbb{P}(M_0\mid{\boldsymbol{x}})> \mathbb{P}(M_1\mid{\boldsymbol{x}}). } The researcher’s conclusion is therefore incorrect: a Bayes factor favouring M_1 does not necessarily make M_1 the more probable model a posteriori.

An IQ threshold

Question page: An IQ threshold

Model population IQ by \mu\sim\operatorname{N}(100,15^2). Conditional on a person’s true IQ \mu, a standardised test score satisfies X\mid\mu\sim\operatorname{N}(\mu,10^2). A candidate obtains x=135, and the classification threshold for “gifted” is 130.

  1. Find an equal-tailed 95\% posterior credible interval for \mu.

  2. For H_0:\mu<130 and H_1:\mu\geq130, calculate the posterior odds \mathbb{P}(H_0\mid x)/\mathbb{P}(H_1\mid x) and state which hypothesis they favour.

  3. Calculate \operatorname{BF}_{01} and reconcile its direction with that of the posterior odds by examining the prior odds.

Question. Model population IQ by \mu\sim\operatorname{N}(100,15^2). Conditional on a person’s true IQ \mu, a standardised test score satisfies X\mid\mu\sim\operatorname{N}(\mu,10^2). A candidate obtains x=135, and the classification threshold for “gifted” is 130.

  1. Find an equal-tailed 95\% posterior credible interval for \mu.

  2. For H_0:\mu<130 and H_1:\mu\geq130, calculate the posterior odds \mathbb{P}(H_0\mid x)/\mathbb{P}(H_1\mid x) and state which hypothesis they favour.

  3. Calculate \operatorname{BF}_{01} and reconcile its direction with that of the posterior odds by examining the prior odds.

Solution. The normal prior and likelihood give posterior precision

\frac1{v_1}=\frac1{15^2}+\frac1{10^2} =\frac1{225}+\frac1{100} =\frac{325}{22500}.

Hence

v_1=\frac{22500}{325}=69.2308.

The posterior mean is \begin{align*} m_1 &=v_1\left(\frac{100}{15^2}+\frac{135}{10^2}\right)\\ &=\frac{15^2(135)+10^2(100)}{15^2+10^2}\\ &=\frac{30375+10000}{325}\\ &=124.2308. \end{align*} Therefore

\boxed{\mu\mid x=135\sim\operatorname{N}(124.2308,69.2308)}.

  1. The posterior standard deviation is \sqrt{69.2308}=8.3205. Thus the equal-tailed interval is \begin{align*} m_1\pm z_{0.975}\sqrt{v_1} &=124.2308\pm1.959964(8.3205)\\ &=\boxed{(107.923,140.539)}. \end{align*}

  2. Under the posterior, \begin{align*} \mathbb{P}(H_0\mid x) &=\mathbb{P}(\mu<130\mid x)\\ &=\Phi\!\left(\frac{130-124.2308}{8.3205}\right)\\ &=0.7560, \end{align*} so \mathbb{P}(H_1\mid x)=1-0.7560=0.2440. The posterior odds are

\boxed{ \frac{\mathbb{P}(H_0\mid x)}{\mathbb{P}(H_1\mid x)} =\frac{0.7560}{0.2440}=3.0977}.

On posterior odds alone, H_0 is favoured by about 3.1:1.

  1. Under the prior, \begin{align*} \mathbb{P}(H_0) &=\mathbb{P}(\mu<130)\\ &=\Phi\!\left(\frac{130-100}{15}\right)\\ &=\Phi(2)=0.97725. \end{align*} The prior odds are therefore

\frac{\mathbb{P}(H_0)}{\mathbb{P}(H_1)} =\frac{0.97725}{1-0.97725} =42.9558.

Consequently \begin{align*} \operatorname{BF}_{01} &=\frac{\text{posterior odds for $H_0$}} {\text{prior odds for $H_0$}}\\ &=\frac{3.0977}{42.9558}\\ &=\boxed{0.0721}. \end{align*} Equivalently, \operatorname{BF}_{10}=1/0.0721\approx13.87: the score is about fourteen times more probable under H_1 than under H_0 after averaging within each hypothesis. The posterior nevertheless favours H_0 because the prior odds favoured it by about 43:1. The Bayes factor records the movement supplied by the data; the posterior odds also contain the starting odds.

Used in

See also

Bayes’ rule · Posterior hypothesis probability · Bayes action for two hypotheses · Marginal likelihood and Bayes factor · A hypothesis region is not a separate model

Glossary and index