Prior odds
Topic of Bayesian hypotheses and model comparison, part of Bayesian statistics
Prior odds
| Type | Topic |
| Taught in | Bayesian hypotheses and model comparison |
| Part | Bayesian statistics |
| Questions using it | 8 |
The ratio \mathbb{P}(H_1)/\mathbb{P}(H_2) of prior probabilities of two hypotheses or models. Bayes’ rule turns it into the posterior odds by multiplying by the Bayes factor.
In Bayesian hypotheses and model comparison
Theorem 1 (Posterior-odds factorisation) For two models M_1 and M_2, Bayes’ rule gives
\mathbb{P}(M_j\mid x) =\frac{p(x\mid M_j)\mathbb{P}(M_j)}{p(x)}.
Taking the ratio cancels the common evidence p(x): \begin{aligned} \frac{\mathbb{P}(M_1\mid x)}{\mathbb{P}(M_2\mid x)} &=\frac{p(x\mid M_1)\mathbb{P}(M_1)/p(x)} {p(x\mid M_2)\mathbb{P}(M_2)/p(x)}\\ &=\underbrace{\frac{p(x\mid M_1)}{p(x\mid M_2)}}_{ \operatorname{BF}_{12}} \underbrace{\frac{\mathbb{P}(M_1)}{\mathbb{P}(M_2)}}_{ \text{prior odds}}. \end{aligned}
Definition 1 (Marginal likelihood and Bayes factor) The marginal likelihood in model M_j is p(x\mid M_j) =\int p(x\mid\theta_j,M_j) p(\theta_j\mid M_j)\,\mathrm{d}\theta_j. It averages the likelihood over the parameter prior; it does not evaluate the likelihood at a fitted maximum. The Bayes factor comparing M_1 with M_2 is \operatorname{BF}_{12}=p(x\mid M_1)/p(x\mid M_2).
A Bayes factor compares whole models, not merely their best-fitting parameter values. The marginal likelihood averages over each model’s prior parameter space, so diffuse priors can spread support across many poor explanations and change the comparison. Equal prior model probabilities are a transparent modelling choice, not a mathematical requirement. Model probabilities, parameter priors, and the set of models under consideration must all be stated before a numerical ratio can be interpreted responsibly.
Remark 1 (A hypothesis region is not a separate model). Two constructions must not be confused:
If H_0 and H_1 are regions of one parameter space, compute one posterior and integrate it over the regions.
If M_1 and M_2 have different dimensions or parameterisations, each model needs its own parameter prior and marginal likelihood. A continuous prior on the larger space assigns probability zero to a lower-dimensional surface.
The fruit example is the first construction. Independence in the advertising table is the second.
Used in
- Taught in Bayesian hypotheses and model comparison
- Also used in Model selection for stochastic models
- Questions that use it (8) — An IQ threshold · Posterior odds (answer) · Q11 (answer) · Q21 (answer) · Q29 (answer) · Q44 (answer) · Question 8 (answer) · Question 18 (answer)
- All 15 questions on Bayesian hypotheses and model comparison
See also
Bayes’ rule · Posterior-odds factorisation · Marginal likelihood and Bayes factor · Posterior hypothesis probability · Bayes action for two hypotheses · A hypothesis region is not a separate model · Probability statements and decisions · Poisson imperfect-fruit example