Nonconjugate priors
Section of The Beta–Binomial model, part of Bayesian statistics
Nonconjugate priors
| Type | Section |
| Taught in | The Beta–Binomial model |
| Part | Bayesian statistics |
Remark 1 (Conjugacy is optional). Conjugacy is convenient, not compulsory. A prior concentrated around 1/2 could be obtained by truncating a Laplace density:
p(\theta)\propto \exp\!\left(-\frac{|\theta-1/2|}{b}\right) \mathbf{1}\{0\leq\theta\leq1\}.
The posterior kernel is then
p(\theta\mid x) \propto \theta^x(1-\theta)^{n-x} \exp\!\left(-\frac{|\theta-1/2|}{b}\right) \mathbf{1}\{0\leq\theta\leq1\}.
If equality rather than proportionality is needed, symmetry supplies the normalising constant without numerical integration: \begin{align*} \int_0^1\exp\!\left(-\frac{|\theta-1/2|}{b}\right)\,\mathrm{d}\theta &=2\int_0^{1/2}\exp\!\left(-\frac{1/2-\theta}{b}\right)\,\mathrm{d}\theta\\ &=2\left[b\exp\!\left(-\frac{1/2-\theta}{b}\right)\right]_{\theta=0}^{1/2}\\ &=2b\left(1-e^{-1/(2b)}\right). \end{align*} Thus the proper truncated-Laplace prior has multiplier ([2b{1-e^{-1/(2b)}}]^{-1}). The resulting posterior is not a named standard density, so its normalising integral and summaries must be evaluated numerically. The later MCMC chapters provide a general route when direct integration becomes difficult.
The difficulty scales with dimension. A one-dimensional posterior can be normalised by quadrature; a direct grid with m points in each of d coordinates requires m^d evaluations. This exponential growth is the computational reason that nonconjugate multiparameter models lead to Monte Carlo methods.
Conjugacy is a computational convenience, not a modelling command. It was especially attractive when integration and simulation were expensive, and it remains valuable because it exposes the update algebra. But if a scientifically sensible prior is nonconjugate, the right response is usually to improve the computation rather than replace the prior merely to obtain a named posterior. Likewise, the MAP can be useful, particularly in optimisation, but one point is not a substitute for the posterior uncertainty that motivated Bayesian analysis in the first place.
Used in
See also
Beta distribution · Beta–Binomial conjugacy · Two pseudo-count conventions · Sequential coherence · Conjugacy is optional · Beta–Binomial posterior prediction · Prior sensitivity · Pseudo-counts and sequential updating