Question 13. Binomial–beta Gibbs sampler

Topic: Markov chains, Gibbs sampling, and Metropolis–Hastings · more questions on this topic
‹ Question 9 Question 16 ›

For x=0,1,\ldots,n and 0\leq\theta\leq 1, with \alpha,\beta>0,

f(x,\theta) = \binom{n}{x}\theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}.

(i) Find and name the conditional distribution of x\mid\theta. The kernel is enough. [4 marks]

(ii) Find and name the conditional distribution of \theta\mid x. The kernel is enough. [4 marks]

(iii) Describe a Gibbs sampler for f(x,\theta). [7 marks]

Worked answer (click to show)

Definitions. The target is a joint distribution of a count X\in\{0,\ldots,n\} and a probability \theta\in(0,1). For positive \alpha,\beta, the kernel factors as a \operatorname{Bin}(n,\theta) mass times a \operatorname{Beta}(\alpha,\beta) kernel. In finding X\mid\theta, normalisation is a sum over counts; for \theta\mid X, it is an integral over (0,1).

The displayed joint expression is a kernel: summing it over x gives \theta^{\alpha-1}(1-\theta)^{\beta-1}, whose integral is B(\alpha,\beta). The normalising factor 1/B(\alpha,\beta) cancels from both full conditionals.

(i) With \theta fixed, \theta^{\alpha-1}(1-\theta)^{\beta-1} is constant in x, but \binom{n}{x} is not. The remaining kernel is \binom{n}{x}\theta^x(1-\theta)^{n-x}. Therefore

x\mid\theta\sim\operatorname{Bin}(n,\theta).

(ii) The factors in \theta are \theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}. Therefore

\theta\mid x\sim\operatorname{Beta}(x+\alpha,\,n-x+\beta).

(iii) Use the two full conditionals in the following order.

The second update uses the new count. Each exact full-conditional update preserves the joint target, so no Metropolis acceptance step is needed; successive recorded pairs are generally dependent. Assess convergence and mixing before using the retained output.

The target also admits a useful simulation check. Its marginal for \theta is \operatorname{Beta}(\alpha,\beta), because summing the binomial mass over x gives one. Drawing \theta from this marginal and then X\mid\theta would generate independent joint pairs directly. The Gibbs algorithm answers the question using full conditionals, but its long-run \theta histogram should agree with that beta marginal.