Pseudo-counts and sequential updating
Section of The Beta–Binomial model, part of Bayesian statistics
Pseudo-counts and sequential updating
| Type | Section |
| Taught in | The Beta–Binomial model |
| Part | Bayesian statistics |
| Questions using it | 4 |
The posterior mean is \begin{align*} \mathbb{E}(\theta\mid x) &=\frac{x+\alpha}{n+\alpha+\beta}\\ &=\frac{n}{n+\alpha+\beta}\frac{x}{n} +\frac{\alpha+\beta}{n+\alpha+\beta} \frac{\alpha}{\alpha+\beta}. \end{align*} Thus \alpha+\beta acts as a prior sample size for the posterior mean, with \alpha prior heads and \beta prior tails.
For the posterior mode, \begin{align*} \widehat\theta_{\mathrm{MAP}} &=\frac{x+\alpha-1}{n+\alpha+\beta-2}\\ &=\frac{n}{n+\alpha+\beta-2}\frac{x}{n} +\frac{\alpha+\beta-2}{n+\alpha+\beta-2} \frac{\alpha-1}{\alpha+\beta-2}, \end{align*} provided the updated parameters exceed one. Hence \alpha-1 and \beta-1 are the natural pseudo-counts for mode calculations. The two conventions do not contradict each other; they correspond to different posterior summaries.
Remark 1 (Two pseudo-count conventions). For posterior means, (\alpha,\beta) act as prior success and failure counts. For an interior posterior mode, the corresponding counts are (\alpha-1,\beta-1). A pseudo-count statement is incomplete unless the posterior summary being interpreted is also named.
Relative to \operatorname{Beta}(1,1), the prior \operatorname{Beta}(5,5) contributes four additional heads and four additional tails in the mode-based convention. That is eight prior trials, not “four heads in four trials.”
Proposition 1 (Sequential coherence) Sequential updating is exact. If data arrive in batches (x_1,n_1) and (x_2,n_2), then \begin{align*} \theta\mid x_1 &\sim\operatorname{Beta}(\alpha+x_1,\beta+n_1-x_1),\\ \theta\mid x_1,x_2 &\sim\operatorname{Beta}(\alpha+x_1+x_2, \beta+n_1+n_2-x_1-x_2), \end{align*} which is the same posterior obtained by combining both batches first.
Used in
- Taught in The Beta–Binomial model
- Also used in Bayesian computation in R · Categorical data and the Dirichlet distribution · Reference formulas and problem-solving workflow
- Questions that use it (4) — The posterior mean is not the observed rate (answer) · A geometric observation and a uniform prior · Geometric success probability (answer) · Inverse-gamma waiting time (answer)
- All 11 questions on The Beta–Binomial model
See also
Two pseudo-count conventions · Beta distribution · Beta–Binomial conjugacy · Sequential coherence · Conjugacy is optional · Beta–Binomial posterior prediction · Prior sensitivity · Nonconjugate priors