Q17. Choosing a conjugate prior

Topic: The Beta–Binomial model · more questions on this topic
‹ Q6 Q42 ›

[3 marks]

Explain what it means for a prior family to be conjugate to a sampling model, and give one practical advantage of using such a family.

A researcher instead chooses a nonconjugate prior because it better represents the available background information. Does this make the Bayesian analysis invalid? Explain what computational difficulty may arise.

Worked answer (click to show)

Definitions. A prior family is conjugate when multiplication by the likelihood and normalisation return a member of that family. For the example below, the Binomial likelihood kernel is \theta^x(1-\theta)^{n-x} and the Beta prior kernel is \theta^{a-1}(1-\theta)^{b-1} on (0,1), with a,b>0. A valid posterior requires a finite, positive normalising integral.

For example, multiplying the Binomial and Beta kernels adds their success and failure exponents. Thus X\mid\theta\sim\operatorname{Bin}(n,\theta), \qquad \theta\sim\operatorname{Beta}(a,b) implies \theta\mid X=x \sim\operatorname{Beta}(a+x,b+n-x). One practical advantage is that posterior updating reduces to updating the hyperparameters. Posterior summaries and predictive calculations can often be obtained analytically, without numerical integration.

A nonconjugate prior does not make the analysis invalid. It may be preferable when it represents the available prior information more faithfully. However, the posterior may not have a standard form, and its normalising constant or required expectations may be difficult to calculate. Numerical quadrature, grid approximation, importance sampling, or MCMC may then be required.