Beta–Binomial posterior prediction
Proposition taught in The Beta–Binomial model, part of Bayesian statistics
Beta–Binomial posterior prediction
| Type | Proposition |
| Taught in | The Beta–Binomial model |
| Part | Bayesian statistics |
| Worked examples | 1 |
If \theta\mid x\sim\operatorname{Beta}(a,b) and, conditionally on \theta, \widetilde X\sim\operatorname{Bin}(m,\theta), then
\mathbb{P}(\widetilde X=k\mid x) =\binom mk\frac{B(k+a,m-k+b)}{B(a,b)}, \qquad k=0,\ldots,m.
For one future Bernoulli observation, \mathbb{P}(\widetilde Y=1\mid x)=a/(a+b).
Examples
Five future tosses
Worked example from The Beta–Binomial model
Suppose the prior is \theta\sim\operatorname{Beta}(2,2) and the observed data contain x=8 heads in n=10 tosses.
Question. Find the posterior predictive probability of a head on the next toss and the probability of at least four heads in five future tosses. Derive the predictive variance for the five-toss count and compare it with the variance obtained by plugging the posterior mean of \theta into a Binomial model.
Solution. The posterior calculation gives
\theta\mid x \sim\operatorname{Beta}(2+8,2+10-8) =\operatorname{Beta}(10,4).
For the very next toss, \begin{align*} \mathbb{P}(\widetilde Y=1\mid x) &=\mathbb{E}(\theta\mid x)\\ &=\frac{10}{10+4}\\ &=\frac57\\ &\approx0.714. \end{align*} The raw sample proportion is 8/10=0.8. The prediction is less extreme because it averages over the posterior produced by the data and the \operatorname{Beta}(2,2) prior.
Now let \widetilde X be the number of heads in m=5 future tosses. With a=10, b=4, and T=a+b=14, \mathbb{P}(\widetilde X=k\mid x) =\binom5k\frac{B(k+a,5-k+b)}{B(a,b)}, \qquad k=0,\ldots,5. For the rising factorial
(c)^{\overline r} =c(c+1)\cdots(c+r-1), \qquad (c)^{\overline 0}=1,
the Beta-function ratio satisfies
\frac{B(k+a,5-k+b)}{B(a,b)} =\frac{(a)^{\overline k}(b)^{\overline{5-k}}} {(a+b)^{\overline5}}.
Therefore the probability of at least four heads is \begin{align*} \mathbb{P}(\widetilde X\geq4\mid x) &=\mathbb{P}(\widetilde X=4\mid x) +\mathbb{P}(\widetilde X=5\mid x)\\ &=\frac{ \binom54(10)^{\overline4}(4)^{\overline1} +\binom55(10)^{\overline5}(4)^{\overline0}} {(14)^{\overline5}}\\ &=\frac{ 5(10\cdot11\cdot12\cdot13)(4) +(10\cdot11\cdot12\cdot13\cdot14)} {14\cdot15\cdot16\cdot17\cdot18}\\ &=\frac{583440}{1028160}\\ &\approx0.567. \end{align*}
This example also shows exactly why posterior prediction is more dispersed than a plug-in Binomial distribution. Conditional on \theta,
\mathbb{E}(\widetilde X\mid\theta)=5\theta, \qquad \operatorname{Var}(\widetilde X\mid\theta)=5\theta(1-\theta).
For \theta\mid x\sim\operatorname{Beta}(a,b) with T=a+b,
\mathbb{E}(\theta\mid x)=\frac aT, \qquad \operatorname{Var}(\theta\mid x)=\frac{ab}{T^2(T+1)}.
Moreover, \begin{align*} \mathbb{E}\{\theta(1-\theta)\mid x\} &=\mathbb{E}(\theta\mid x)-\mathbb{E}(\theta^2\mid x)\\ &=\frac aT-\frac{a(a+1)}{T(T+1)}\\ &=\frac{a(T+1)-a(a+1)}{T(T+1)}\\ &=\frac{a(T-a)}{T(T+1)}\\ &=\frac{ab}{T(T+1)}. \end{align*} Applying Equation (3) in Bayesian foundations, \begin{align*} \operatorname{Var}(\widetilde X\mid x) &=\mathbb{E}\{5\theta(1-\theta)\mid x\} +\operatorname{Var}(5\theta\mid x)\\ &=\frac{5ab}{T(T+1)} +\frac{25ab}{T^2(T+1)}\\ &=\frac{5ab(T+5)}{T^2(T+1)}\\ &=\frac{5(10)(4)(19)}{14^2(15)}\\ &=\frac{190}{147}\\ &\approx1.293. \end{align*} If one instead plugs in the posterior mean \widehat\theta=10/14, the variance would be only
5\widehat\theta(1-\widehat\theta) =5\left(\frac{10}{14}\right)\left(\frac4{14}\right) =\frac{50}{49} \approx1.020.
The difference is the second term in Equation (3) in Bayesian foundations: uncertainty about \theta.
Used in
- Taught in The Beta–Binomial model, where the derivation and examples are.
- All 11 questions on The Beta–Binomial model
See also
Beta distribution · Beta–Binomial conjugacy · Two pseudo-count conventions · Sequential coherence · Conjugacy is optional