Q27. Machine downtime and a Gamma prior
[25 marks]
Suppose the proportion of time X that a production machine is down during its daily shift follows a \operatorname{Beta}(1,\theta) distribution: f(x\mid\theta)=\theta(1-x)^{\theta-1}, \qquad 0<x<1,\quad\theta>0. Let X_i denote the proportion of downtime on day i, for i=1,\ldots,n, and assume the observations are conditionally iid given \theta. Use the \operatorname{Gamma}(\alpha,\beta) prior in shape–rate form, p(\theta)=\frac{\beta^\alpha}{\Gamma(\alpha)} \theta^{\alpha-1}e^{-\beta\theta}, \qquad\theta>0,\quad\alpha,\beta>0.
(a) [1 mark] State, without derivation, the prior mean.
(b) [8 marks] Obtain an expression for the posterior distribution of \theta given the data {\boldsymbol{x}}=(x_1,\ldots,x_n), and identify its distribution.
(c) [2 marks] State, without derivation, the posterior mean.
(d) [9 marks] Explain what happens to the posterior mean:
(i) as n\to\infty, with the prior fixed;
(ii) as \alpha,\beta\to0, with the data fixed;
(iii) as \alpha,\beta\to\infty, with the data fixed.
State any additional conditions required for a definite limit.
(e) [5 marks] Interpret the posterior mean by completing:
“The effect of the prior hyperparameters \alpha and \beta on the posterior mean of \theta is like …”
Support your interpretation with an appropriate expression for the posterior mean.
Definitions. The sampling density \theta(1-x)^{\theta-1} is \operatorname{Beta}(1,\theta) on (0,1). The prior \operatorname{Gamma}(\alpha,\beta) uses shape \alpha>0 and rate \beta>0, so its kernel is \theta^{\alpha-1}e^{-\beta\theta} and its mean is \alpha/\beta. The observations are conditionally iid with n\ge1. The transformed sum S=-\sum_i\log(1-x_i) is positive and is not the raw total downtime.
(a) Prior mean. The prior is Gamma in shape–rate form, so \boxed{\mathbb{E}(\theta)=\frac{\alpha}{\beta}.}
(b) Posterior distribution. For conditionally iid observations, the likelihood is \begin{aligned} L(\theta;{\boldsymbol{x}}) &=\prod_{i=1}^{n}\theta(1-x_i)^{\theta-1}\\ &=\theta^n\prod_{i=1}^{n}(1-x_i)^{\theta-1}\\ &=\theta^n\exp\!\left\{(\theta-1) \sum_{i=1}^{n}\log(1-x_i)\right\}. \end{aligned} Define the positive statistic S=-\sum_{i=1}^{n}\log(1-x_i)>0. Then L(\theta;{\boldsymbol{x}}) =e^S\theta^ne^{-S\theta} \propto\theta^ne^{-S\theta},\qquad\theta>0, where only e^S, which does not involve \theta, has been discarded. Multiplying by the Gamma prior gives \begin{aligned} p(\theta\mid{\boldsymbol{x}}) &\propto L(\theta;{\boldsymbol{x}})p(\theta)\\ &\propto\theta^ne^{-S\theta} \theta^{\alpha-1}e^{-\beta\theta}\\ &=\theta^{\alpha+n-1}e^{-(\beta+S)\theta}. \end{aligned} Matching the shape–rate Gamma kernel gives \boxed{\theta\mid{\boldsymbol{x}}\sim \operatorname{Gamma}(\alpha+n,\beta+S),\qquad\theta>0.} The normalised posterior density is \boxed{p(\theta\mid{\boldsymbol{x}}) =\frac{(\beta+S)^{\alpha+n}}{\Gamma(\alpha+n)} \theta^{\alpha+n-1}e^{-(\beta+S)\theta},\qquad\theta>0.} Equivalently, the posterior rate is \beta-\sum_i\log(1-x_i). It is positive: each \log(1-x_i) is negative.
(c) Posterior mean. \boxed{\mathbb{E}(\theta\mid{\boldsymbol{x}})=\frac{\alpha+n}{\beta+S} =\frac{\alpha+n}{\beta-\sum_{i=1}^{n}\log(1-x_i)}.}
(d) Limiting behaviour. First identify the maximum-likelihood estimate. From the log-likelihood \ell(\theta)=n\log\theta-S\theta+\text{constant}, we have \ell'(\theta)=\frac n\theta-S,\qquad \ell''(\theta)=-\frac n{\theta^2}<0. Consequently, \widehat\theta_{\mathrm{MLE}}=\frac nS.
(i) Increasing sample size with a fixed prior. Write S_n=-\sum_{i=1}^{n}\log(1-X_i) to make its dependence on n explicit. The posterior mean is \frac{\alpha+n}{\beta+S_n} =\frac{1+\alpha/n}{S_n/n+\beta/n}. If the data arise from the stated model at a true value \theta_0>0, then Z_i=-\log(1-X_i) are iid \operatorname{Exp}(\theta_0) variables. For z\geq0, \mathbb{P}(Z_i>z\mid\theta_0) =\mathbb{P}(X_i>1-e^{-z}\mid\theta_0)=e^{-\theta_0z}. Thus \mathbb{E}(Z_i\mid\theta_0)=1/\theta_0, and the law of large numbers gives S_n/n\to1/\theta_0. Hence \boxed{\mathbb{E}(\theta\mid X_1,\ldots,X_n)\longrightarrow\theta_0.} The MLE n/S_n has the same limit, and the fixed prior’s influence vanishes. It would be incorrect to let n increase while holding S_n fixed in this sampling argument: new observations also change the sum.
(ii) \alpha,\beta\to0 with the data fixed. For n\geq1 and observed 0<x_i<1, we have S>0. Therefore \boxed{\frac{\alpha+n}{\beta+S}\longrightarrow\frac nS =\widehat\theta_{\mathrm{MLE}}.} The proper Gamma priors have the formal limiting kernel 1/\theta, not a uniform density; the resulting limiting posterior is the proper \operatorname{Gamma}(n,S) distribution.
(iii) \alpha,\beta\to\infty with the data fixed. Compare the posterior mean with the prior mean: \frac{(\alpha+n)/(\beta+S)}{\alpha/\beta} =\frac{1+n/\alpha}{1+S/\beta}\longrightarrow1. Thus the posterior mean follows the prior mean asymptotically. To claim a specific finite limit, specify the limiting ratio. If \alpha/\beta\to m_0\in(0,\infty), then \boxed{\mathbb{E}(\theta\mid{\boldsymbol{x}})\longrightarrow m_0.} In particular, this holds when the prior mean is kept fixed at m_0 while its concentration increases. Both prior and posterior variances then tend to zero. Without a condition on \alpha/\beta, there is no unique limit: \alpha=\beta\to\infty gives limit 1, whereas \alpha=2\beta\to\infty gives limit 2; other paths can diverge or fail to converge.
(e) Interpreting the prior’s effect. The posterior mean can be written exactly as \begin{aligned} \mathbb{E}(\theta\mid{\boldsymbol{x}}) &=\frac{\alpha+n}{\beta+S}\\ &=\frac{\beta}{\beta+S}\frac{\alpha}{\beta} +\frac{S}{\beta+S}\frac nS. \end{aligned} Hence \boxed{\mathbb{E}(\theta\mid{\boldsymbol{x}}) =w\,\mathbb{E}(\theta)+(1-w)\widehat\theta_{\mathrm{MLE}}, \qquad w=\frac{\beta}{\beta+S}.} Since 0<w<1 and the weights sum to one, the posterior mean lies between the prior mean and the MLE.
An exam-style completion of the sentence is:
“The effect of the prior hyperparameters \alpha and \beta on the posterior mean of \theta is like adding prior information that pulls the data-based estimate n/S toward the prior mean \alpha/\beta, with prior weight \beta/(\beta+S).”
Keeping \alpha/\beta fixed while increasing \alpha,\beta increases the prior weight. Increasing the amount of data increases S under the model and reduces that weight.
There is also a pseudo-data interpretation on the transformed scale Z_i=-\log(1-X_i), where Z_i\mid\theta\sim\operatorname{Exp}(\theta). The likelihood estimate is the observation count divided by total transformed exposure, n/S. In the posterior-mean formula, \alpha adds to the count and \beta adds to the transformed exposure: \frac{n+\text{prior pseudo-count}}{S+\text{prior exposure}} =\frac{n+\alpha}{S+\beta}. These are contributions to the posterior mean, not necessarily a literal integer number of extra observations. This interpretation concerns the transformed downtime, not the raw sum \sum_i x_i.
Key result. The posterior is Gamma with shape \alpha+n and rate \beta-\sum_i\log(1-x_i). A strong prior dominates only toward the mean specified by its ratio \alpha/\beta.