Prior sensitivity

Section of The Beta–Binomial model, part of Bayesian statistics

Prior sensitivity

Type	Section
Taught in	The Beta–Binomial model
Part	Bayesian statistics

For n=10 tosses and x=3 heads, these three priors give \begin{align*} \operatorname{Beta}(5,5)&\longrightarrow\operatorname{Beta}(8,12),\\ \operatorname{Beta}(1,1)&\longrightarrow\operatorname{Beta}(4,8),\\ \operatorname{Beta}(0.6,0.6)&\longrightarrow\operatorname{Beta}(3.6,7.6). \end{align*} For example, under the uniform prior,

\mathbb{E}(\theta\mid x)=\frac4{12}=\frac13, \qquad \widehat\theta_{\mathrm{MAP}} =\frac{4-1}{4+8-2}=\frac3{10}=0.3.

The posterior probability of downward bias is

\mathbb{P}(\theta<0.5\mid x) =\int_0^{0.5}\frac{\theta^3(1-\theta)^7}{B(4,8)}\,\mathrm{d}\theta.

With n=100 and x=30 heads, the same priors give \begin{align*} \operatorname{Beta}(5,5)&\longrightarrow\operatorname{Beta}(35,75),\\ \operatorname{Beta}(1,1)&\longrightarrow\operatorname{Beta}(31,71),\\ \operatorname{Beta}(0.6,0.6)&\longrightarrow\operatorname{Beta}(30.6,70.6). \end{align*} The relative effect of the prior is smaller because the data contribute 100 counts while the first prior contributes a fixed amount.

Figure 1: The three priors, then the posteriors they produce after 3 heads in 10 tosses and after 30 heads in 100. Both datasets have the same observed proportion 0.3, marked by the dashed vertical line, so moving to the right changes only how much weight the data carry. Ten tosses leave three visibly different posteriors; one hundred leave three that have nearly closed on each other and on x/n. Disagreement about the prior is a disagreement about small samples. The \operatorname{Beta}(0.6,0.6) prior is unbounded at 0 and 1, so its curve leaves the top of the first panel.

Parameterisation check

Direct substitution of x=30 in Equation (4) in The Beta–Binomial model gives the first shape parameters 35, 31, and 30.6 written above. The values 36, 32, and 31.6 are one unit larger and do not follow from that substitution. The second shape parameters that match x=30 are 75, 71, and 70.6.

Used in

• Taught in The Beta–Binomial model
• Also used in Bayesian computation in R
• All 11 questions on The Beta–Binomial model

See also

Beta distribution · Beta–Binomial conjugacy · Two pseudo-count conventions · Sequential coherence · Conjugacy is optional · Beta–Binomial posterior prediction · Pseudo-counts and sequential updating · Nonconjugate priors

Glossary and index