Question 6. Geometric–beta Gibbs sampler

Topic: Markov chains, Gibbs sampling, and Metropolis–Hastings · more questions on this topic
‹ Q59 Question 9 ›

Consider X and Y with joint density

f(x,y)\propto y^{\alpha}(1-y)^{x+\beta-1},

for x=1,2,3,\ldots and 0<y<1, where \alpha>0 and \beta>0. Describe a Gibbs sampler for this joint density.

Definitions. This target has one discrete coordinate X\in\{1,2,\ldots\} and one continuous coordinate Y\in(0,1). Here \operatorname{Geom}(p) counts trials up to and including the first success, with mass p(1-p)^{x-1}. A \operatorname{Beta}(a,b) density is proportional to y^{a-1}(1-y)^{b-1}. When finding a full conditional, keep all factors involving the coordinate being updated and normalise over that coordinate’s support.

For 0<y<1, the factors in x are (1-y)^{x-1}. For r=1-y\in(0,1),

\sum_{x=1}^{\infty}r^{x-1}=\frac{1}{1-r}=\frac{1}{y},

so

X\mid y\sim\operatorname{Geom}(y), \qquad \mathbb{P}(X=x\mid y)=y(1-y)^{x-1},\quad x=1,2,\ldots.

The factors in y are y^{(\alpha+1)-1}(1-y)^{(x+\beta)-1}, so

Y\mid x\sim\operatorname{Beta}(\alpha+1,\,x+\beta).

Record the completed pair. Each draw is from an exact full conditional, so there is no acceptance step. To see why an update preserves the joint target, suppose a pair already has density f: leaving Y unchanged and redrawing X from f(x\mid Y) gives f_Y(y)f(x\mid y)=f(x,y) again. The same reasoning applies to the other coordinate and hence to their composition. Convergence from the chosen start still needs assessment, and successive sweeps are generally dependent.

The target is proper: summing its kernel over x leaves y^{\alpha-1}(1-y)^\beta, which has a finite beta integral for the stated positive parameters. In fact, this also identifies Y\sim\operatorname{Beta}(\alpha,\beta+1) marginally. It provides a useful check on a Gibbs simulation; it is different from the full conditional Y\mid X.

Support and mixing checks. Correct full conditionals alone do not guarantee useful exploration. For example, take a target uniform on the two disjoint squares (0,1)^2 and (2,3)^2. Updating one coordinate while holding the other fixed cannot leave the current square, so a single-coordinate Gibbs chain cannot explore the whole target. The present geometric–beta target is positive over its product support and has no such hard separation. Strong dependence can nevertheless make exploration slow; a longer discarded prefix does not fix poor mixing. If implementing the first draw with R’s rgeom, add one because R counts failures before success.