Probability statements and decisions
Section of Bayesian hypotheses and model comparison, part of Bayesian statistics
Probability statements and decisions
| Type | Section |
| Taught in | Bayesian hypotheses and model comparison |
| Part | Bayesian statistics |
Definition 1 (Posterior hypothesis probability) Suppose a parameter space \Omega is partitioned into regions \Omega_0 and \Omega_1. A test that instead reports a long-run p-value under a null sampling model is a hypothesis test. Numerical examples of those tests are in Testing examples. The posterior hypothesis probabilities are \begin{align*} \mathbb{P}(H_0\mid x) &=\mathbb{P}(\theta\in\Omega_0\mid x) =\int_{\Omega_0}p(\theta\mid x)\,\mathrm{d}\theta,\\ \mathbb{P}(H_1\mid x) &=\mathbb{P}(\theta\in\Omega_1\mid x) =\int_{\Omega_1}p(\theta\mid x)\,\mathrm{d}\theta. \end{align*} If the two regions exhaust the support, the probabilities sum to one.
Proposition 1 (Bayes action for two hypotheses) These probabilities do not by themselves define an action. If choosing action a_0 when H_1 is true costs L_{01} and choosing a_1 when H_0 is true costs L_{10}, the posterior expected losses are
\rho(a_0\mid x)=L_{01}\mathbb{P}(H_1\mid x), \qquad \rho(a_1\mid x)=L_{10}\mathbb{P}(H_0\mid x).
Choose a_0 when \begin{align*} L_{01}\mathbb{P}(H_1\mid x) &<L_{10}\mathbb{P}(H_0\mid x)\\ \Longleftrightarrow\quad \frac{\mathbb{P}(H_0\mid x)}{\mathbb{P}(H_1\mid x)} &>\frac{L_{01}}{L_{10}}. \end{align*} Equal losses reduce the threshold to one. This separates the probability calculation from the decision rule.
The Bayesian calculation need not force an all-or-nothing verdict. A useful practice is to report the posterior support for each hypothesis and to let the action depend on the real consequences of being wrong. If a decision is compulsory, losses provide the threshold. Reporting evidence and choosing an action are related tasks, but they are not the same task. This is also why Bayesians are often less enthusiastic about compulsory model selection than about reporting posterior model probabilities.
In frequentist design, type-I and type-II error rates \alpha and \beta are properties of repeated use of a test. Values such as 0.05 and 0.20 are planning choices, not scientific constants, and sample-size calculations also require an effect size worth detecting.
Used in
See also
Posterior hypothesis probability · Bayes action for two hypotheses · Posterior-odds factorisation · Marginal likelihood and Bayes factor · A hypothesis region is not a separate model · Poisson imperfect-fruit example · Bayes factors and posterior odds · Bayes factor for independence in the 2×3 table