Question 11. Geometric success probability

Topic: Exponential families and conjugacy · more questions on this topic
‹ Question 10 Question 14 ›

Let x_1,\ldots,x_n be iid with

f(x\mid p)=(1-p)^{x}p, \quad x=0,1,2,\ldots,

and p\sim\operatorname{Beta}(\alpha,\beta).

(i) Find the posterior distribution of p and identify it. [3 marks]

(ii) Is this prior conjugate? [1 mark]

(iii) Show that a \operatorname{Beta}(a,b) mean equals a/(a+b). [2 marks]

(iv) Hence show that \mathbb{E}(p\mid{\boldsymbol{x}})=(\alpha+n)/(\alpha+\beta+n+n\bar x). [1 mark]

(v) What happens to the posterior mean as n\to\infty? [1 mark]

(vi) What happens if \alpha,\beta\to 0? [1 mark]

(vii) Complete: “The effect of the prior hyperparameters \alpha,\beta on the posterior mean of p is like…” [2 marks]

Worked answer (click to show)

Definitions. Here X_i counts failures before the first success, so its support begins at zero. Each observation represents one success and x_i failures. Put F=\sum_i x_i=n\bar x. The beta density is p^{\alpha-1}(1-p)^{\beta-1}/B(\alpha,\beta) on 0<p<1. This geometric convention differs from the trial-count convention in Question 6 by one.

(i) The likelihood is p^{n}(1-p)^{\sum x_i}. The beta prior contributes p^{\alpha-1}(1-p)^{\beta-1}, so

\boxed{p\mid{\boldsymbol{x}}\sim\operatorname{Beta}\bigl(\alpha+n,\;\beta+\textstyle\sum_{i=1}^{n}x_i\bigr).}

(ii) Yes. The posterior is beta, the same family as the prior.

(iii) For a>0 and b>0, the expectation integral is

\mathbb{E}(P)=\frac1{B(a,b)}\int_0^1 p^a(1-p)^{b-1}\,dp =\frac{B(a+1,b)}{B(a,b)}.

Using B(a,b)=\Gamma(a)\Gamma(b)/\Gamma(a+b) and \Gamma(a+1)=a\Gamma(a),

\frac{B(a+1,b)}{B(a,b)} = \frac{\Gamma(a+1)\Gamma(a+b)}{\Gamma(a)\Gamma(a+b+1)} = \frac{a}{a+b},

and this ratio is the beta mean.

(iv) By (i) and (iii),

\mathbb{E}(p\mid{\boldsymbol{x}}) = \frac{\alpha+n}{\alpha+\beta+n+n\bar x}.

(v) Divide the numerator and denominator by n:

\mathbb{E}(p\mid{\boldsymbol{x}})=\frac{1+\alpha/n}{1+\bar x+(\alpha+\beta)/n}.

For fixed prior parameters this approaches the data-based estimate 1/(1+\bar x). The log-likelihood is n\log p+F\log(1-p), and setting its derivative to zero gives \hat p=n/(n+F) when F>0. If the data come from the model at p_0, then \bar X_n\to(1-p_0)/p_0 and both estimates converge to p_0.

(vi) If \alpha,\beta\to 0, the posterior mean tends to n/(n+n\bar x)=1/(1+\bar x). The prior drops out.

This is a limit of proper beta priors, not substitution of \alpha=\beta=0 into a proper prior density. If F=0, the limiting estimate is the boundary value 1 and the limiting beta kernel is not a proper density; the posteriors for positive hyperparameters remain proper.

(vii) The effect is like \alpha prior successes and \beta prior failures, added to the n observed successes and \sum x_i observed failures.

An exact weighted-average expression makes that interpretation explicit:

\mathbb{E}(p\mid{\boldsymbol{x}}) =\frac{\alpha+\beta}{\alpha+\beta+n+F}\frac{\alpha}{\alpha+\beta} +\frac{n+F}{\alpha+\beta+n+F}\frac{n}{n+F}.

The beta prior mean and the observed success fraction are weighted by their respective total counts. These pseudo-counts describe the posterior-mean update; the powers in the prior density itself are \alpha-1 and \beta-1.