Q2. Gamma waiting times with an inverse-Gamma prior
At each of n locations, researchers wait until they capture the mth animal. Let X_i denote the waiting time at location i, and suppose X_i\mid\theta \overset{\mathrm{iid}}{\sim}\operatorname{Gamma}(m,\theta), \qquad i=1,\ldots,n, where the Gamma distribution is parameterised by shape m and scale \theta, so that f(x_i\mid\theta) =\frac{x_i^{m-1}}{\Gamma(m)\theta^m} \exp\left(-\frac{x_i}{\theta}\right), \qquad x_i>0. Assume the prior distribution \theta\sim\operatorname{InvGamma}(\alpha_0,\beta_0), with density p(\theta) =\frac{\beta_0^{\alpha_0}}{\Gamma(\alpha_0)} \theta^{-\alpha_0-1} \exp\left(-\frac{\beta_0}{\theta}\right), \qquad \theta>0.
(a) Write down the likelihood L(\theta;{\boldsymbol{x}}), up to proportionality in \theta.
(b) Find the posterior distribution of \theta\mid{\boldsymbol{x}}.
(c) State the posterior mean \mathbb{E}(\theta\mid{\boldsymbol{x}}) and the condition under which it exists.
(d) Suppose \widehat{\theta}_{\mathrm{MLE}}=\frac{\bar{x}}{m}. Show that the posterior mean is a weighted average of the prior mean and the MLE. Verify that the weights sum to one and briefly interpret the result.
(e) Examine the limiting behaviour of the posterior mean:
(i) What happens to the weights as m\to\infty?
(ii) What happens, formally, as the prior becomes increasingly weak, represented by \alpha_0,\beta_0\to0?
Definitions. Here \operatorname{Gamma}(m,\theta) uses shape m and scale \theta, not rate. Write \bar x=n^{-1}\sum_i x_i and \widehat\theta_{\mathrm{MLE}}=\bar x/m. Our inverse-Gamma convention is p(\theta)\propto\theta^{-a-1}e^{-b/\theta} for \theta>0 and a,b>0. The observations are conditionally iid, m>0 is known, and the prior has \alpha_0,\beta_0>0. Conditions for the means to exist are checked below.
(a) Likelihood
Independence gives \begin{aligned} L(\theta;{\boldsymbol{x}}) &=\prod_{i=1}^n f(x_i\mid\theta)\\ &=\prod_{i=1}^n \frac{x_i^{m-1}}{\Gamma(m)\theta^m} \exp\left(-\frac{x_i}{\theta}\right). \end{aligned} Discarding factors that do not depend on \theta, \begin{aligned} L(\theta;{\boldsymbol{x}}) &\propto \prod_{i=1}^n \theta^{-m}\exp\left(-\frac{x_i}{\theta}\right)\\ &=\theta^{-nm} \exp\left(-\frac{\sum_{i=1}^n x_i}{\theta}\right). \end{aligned} Therefore, \boxed{ L(\theta;{\boldsymbol{x}}) \propto \theta^{-nm} \exp\left(-\frac{\sum_{i=1}^n x_i}{\theta}\right) }.
(b) Posterior distribution
The inverse-Gamma prior has kernel p(\theta) \propto \theta^{-\alpha_0-1} \exp\left(-\frac{\beta_0}{\theta}\right). By Bayes’ rule, \begin{aligned} p(\theta\mid{\boldsymbol{x}}) &\propto L(\theta;{\boldsymbol{x}})p(\theta)\\ &\propto \theta^{-nm} \exp\left(-\frac{\sum_{i=1}^n x_i}{\theta}\right) \theta^{-\alpha_0-1} \exp\left(-\frac{\beta_0}{\theta}\right)\\ &\propto \theta^{-(\alpha_0+nm)-1} \exp\left( -\frac{\beta_0+\sum_{i=1}^n x_i}{\theta} \right). \end{aligned} This is an inverse-Gamma kernel, so \boxed{ \theta\mid{\boldsymbol{x}} \sim \operatorname{InvGamma}\left( \alpha_0+nm, \beta_0+\sum_{i=1}^n x_i \right) }. Equivalently, the conjugate update is \boxed{ \alpha_0\longmapsto\alpha_0+nm, \qquad \beta_0\longmapsto\beta_0+\sum_{i=1}^n x_i }.
(c) Posterior mean
Under the parameterisation used here, if Z\sim\operatorname{InvGamma}(a,b), then \mathbb{E}(Z)=\frac{b}{a-1}, \qquad a>1. It follows that \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}}) =\frac{\beta_0+\sum_{i=1}^n x_i} {\alpha_0+nm-1} }, \qquad \boxed{\alpha_0+nm>1}.
(d) Weighted-average interpretation
Provided \alpha_0>1, the prior mean is \mathbb{E}(\theta)=\frac{\beta_0}{\alpha_0-1}, and hence \beta_0=(\alpha_0-1)\mathbb{E}(\theta). Also, \sum_{i=1}^n x_i =n\bar{x} =nm\widehat{\theta}_{\mathrm{MLE}}. Substituting these identities into the posterior mean gives \begin{aligned} \mathbb{E}(\theta\mid{\boldsymbol{x}}) &=\frac{(\alpha_0-1)\mathbb{E}(\theta) +nm\widehat{\theta}_{\mathrm{MLE}}} {\alpha_0+nm-1}\\ &=\frac{\alpha_0-1}{\alpha_0+nm-1}\mathbb{E}(\theta) +\frac{nm}{\alpha_0+nm-1} \widehat{\theta}_{\mathrm{MLE}}. \end{aligned} Thus, \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}}) =w_0\mathbb{E}(\theta)+w_n\widehat{\theta}_{\mathrm{MLE}} }, where w_0=\frac{\alpha_0-1}{\alpha_0+nm-1}, \qquad w_n=\frac{nm}{\alpha_0+nm-1}. The weights are non-negative when \alpha_0>1, and w_0+w_n =\frac{\alpha_0-1+nm}{\alpha_0+nm-1} =1. The posterior mean is therefore a convex combination of the prior mean and the MLE. This is Bayesian shrinkage: the posterior estimate balances prior information against information in the observed data.
(e) Limiting behaviour
(i) As m\to\infty.
For fixed n and fixed prior hyperparameters, w_0=\frac{\alpha_0-1}{\alpha_0+nm-1}\longrightarrow0, \qquad w_n=\frac{nm}{\alpha_0+nm-1}\longrightarrow1. Hence the prior receives vanishing weight. If the MLE remains bounded (as it does in probability under sampling at a fixed true scale), then \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}}) -\widehat{\theta}_{\mathrm{MLE}} \longrightarrow0 }. In this sense, as the data contribution dominates, the posterior mean approaches the data-based estimate.
(ii) As the prior becomes weak.
Starting from the direct expression for the posterior mean and formally letting \alpha_0,\beta_0\to0 gives \begin{aligned} \mathbb{E}(\theta\mid{\boldsymbol{x}}) &\longrightarrow \frac{\sum_{i=1}^n x_i}{nm-1}\\ &=\frac{nm}{nm-1}\widehat{\theta}_{\mathrm{MLE}}, \qquad nm>1. \end{aligned} Therefore, \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}}) \longrightarrow \frac{nm}{nm-1}\widehat{\theta}_{\mathrm{MLE}} \approx\widehat{\theta}_{\mathrm{MLE}} }, with the approximation becoming exact as nm\to\infty.
This is a formal limiting argument. A proper inverse-Gamma prior has \alpha_0,\beta_0>0, and its mean exists only when \alpha_0>1. Thus the path \alpha_0,\beta_0\to0 does not remain within the class of proper priors having finite means; the limit should be taken from the posterior expression, not from the weighted-average representation involving \mathbb{E}(\theta).
Key result. The full progression is \boxed{ L(\theta;{\boldsymbol{x}}) \longrightarrow p(\theta\mid{\boldsymbol{x}}) \longrightarrow \mathbb{E}(\theta\mid{\boldsymbol{x}}) \longrightarrow w_0\mathbb{E}(\theta)+w_n\widehat{\theta}_{\mathrm{MLE}} \longrightarrow \widehat{\theta}_{\mathrm{MLE}} }, as the data contribution dominates.