Q13. Bayesian German tank problem
In the Second World War, Allied statisticians estimated German tank production from the serial numbers on captured tanks. Treat the serial numbers as continuous. Suppose X_1,\ldots,X_n\mid\theta \overset{\mathrm{iid}}{\sim}\operatorname{Uniform}(0,\theta), \qquad \theta>0, and write x_{(n)}=\max_i x_i. Place the Pareto prior p(\theta\mid\alpha,\beta) =\frac{\alpha\beta^\alpha}{\theta^{\alpha+1}}, \qquad \theta\geq\beta, where \alpha,\beta>0. Here \beta is a known lower bound on production.
[13 marks]
(a) Write the likelihood with every indicator function shown explicitly. Derive the posterior distribution of \theta and show that it is Pareto. State its parameters, and state exactly when the data change the prior lower bound. [4 marks]
(b) Find the posterior mean. The MLE is \widehat{\theta}_{\mathrm{MLE}}=x_{(n)}. Explain why the posterior mean always exceeds it, and why this is sensible rather than a sign of bias in the Bayesian estimate. [2 marks]
(c) Derive the posterior predictive density of one new captured serial number \widetilde X. Show that it is a two-component mixture, and identify both components and their weights. Find \mathbb{P}\!\left( \widetilde X>\max\{\beta,x_{(n)}\}\mid{\boldsymbol{x}} \right) and interpret it. [4 marks]
(d) The Pitman–Koopman–Darmois theorem links sufficient statistics of fixed dimension to exponential families. Here x_{(n)} is a one-dimensional sufficient statistic for every n, and a conjugate prior exists. Explain why this does not make \operatorname{Uniform}(0,\theta) an exponential family, and why it does not contradict the theorem. [2 marks]
(e) The captured serials are 19,40,42,60, and the prior has \alpha=1 and \beta=50. Give the posterior mean and a 95% highest-posterior-density interval for \theta. Justify the lower endpoint of the interval. [1 mark]
Definitions. Write x_{(1)}=\min_i x_i and x_{(n)}=\max_i x_i; \mathbf1\{A\} is one when A holds and zero otherwise. Our Pareto convention is p(\theta)=ab^a\theta^{-a-1}\mathbf1\{\theta\ge b\}, with shape a>0 and lower bound b>0. \widetilde X denotes a new serial number, conditionally independent of the sample given \theta. We use the stated continuous, conditionally iid model with n\ge1 and positive observations.
Write a_n=\alpha+n, \qquad b_n=\max\{\beta,x_{(n)}\}.
(a) Likelihood and posterior
For observations from \operatorname{Uniform}(0,\theta), \begin{aligned} L(\theta;{\boldsymbol{x}}) &=\prod_{i=1}^n \left[\frac{1}{\theta}\, \mathbf{1}\{0<x_i<\theta\}\right]\\ &=\theta^{-n}\, \mathbf{1}\{x_{(1)}>0\} \prod_{i=1}^n\mathbf{1}\{x_i<\theta\}\\ &=\theta^{-n}\, \mathbf{1}\{x_{(1)}>0\} \mathbf{1}\{x_{(n)}<\theta\}. \end{aligned} For a valid observed sample, \mathbf{1}\{x_{(1)}>0\} does not depend on \theta. Hence L(\theta;{\boldsymbol{x}}) \propto\theta^{-n}\mathbf{1}\{\theta>x_{(n)}\}. Multiplying by the prior gives \begin{aligned} p(\theta\mid{\boldsymbol{x}}) &\propto \theta^{-n}\mathbf{1}\{\theta>x_{(n)}\} \theta^{-(\alpha+1)}\mathbf{1}\{\theta\geq\beta\}\\ &=\theta^{-(\alpha+n+1)} \mathbf{1}\{\theta\geq b_n\}\\ &=\theta^{-(a_n+1)}\mathbf{1}\{\theta\geq b_n\}. \end{aligned} The distinction between strict and non-strict inequalities at one endpoint is irrelevant for a continuous density. Since \int_{b_n}^{\infty}\theta^{-(a_n+1)}\,d\theta =\frac{b_n^{-a_n}}{a_n}, the normalised posterior density is p(\theta\mid{\boldsymbol{x}}) =\frac{a_n b_n^{a_n}}{\theta^{a_n+1}}, \qquad \theta\geq b_n. Therefore, \boxed{ \theta\mid{\boldsymbol{x}} \sim\operatorname{Pareto}\!\left( \alpha+n,\max\{\beta,x_{(n)}\} \right). } The data change the lower bound exactly when x_{(n)}>\beta. Otherwise the lower bound remains \beta, although the shape still changes from \alpha to \alpha+n.
(b) Posterior mean
Because a_n=\alpha+n>1, the posterior mean exists and is \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}}) =\frac{a_n b_n}{a_n-1} =\frac{(\alpha+n)\max\{\beta,x_{(n)}\}} {\alpha+n-1}. } Since a_n/(a_n-1)>1 and b_n\geq x_{(n)}, \mathbb{E}(\theta\mid{\boldsymbol{x}})>b_n\geq x_{(n)} =\widehat{\theta}_{\mathrm{MLE}}. This is sensible. The MLE is the smallest value of \theta compatible with the data because the likelihood decreases for \theta>x_{(n)}. Moreover, X_{(n)}<\theta almost surely and \mathbb{E}(X_{(n)}\mid\theta)=\frac{n}{n+1}\theta, so the sample maximum systematically falls below the true upper endpoint. The posterior mean allows for the unobserved region above the largest captured serial. Frequentist bias is a repeated-sampling property; the posterior mean is instead the Bayes estimate under squared-error loss.
(c) Posterior predictive distribution
For a new serial number \widetilde X, \begin{aligned} p(\widetilde x\mid{\boldsymbol{x}}) &=\int_{b_n}^{\infty} \frac{1}{\theta}\mathbf{1}\{0<\widetilde x<\theta\} \frac{a_n b_n^{a_n}}{\theta^{a_n+1}}\,d\theta\\ &=a_n b_n^{a_n} \int_{\max\{b_n,\widetilde x\}}^{\infty} \theta^{-(a_n+2)}\,d\theta, \qquad \widetilde x>0. \end{aligned} Thus p(\widetilde x\mid{\boldsymbol{x}}) = \begin{cases} \displaystyle \frac{a_n}{a_n+1}\frac{1}{b_n}, &0<\widetilde x\leq b_n,\\[1.1em] \displaystyle \frac{a_n b_n^{a_n}}{(a_n+1)\widetilde x^{a_n+1}}, &\widetilde x>b_n,\\[0.8em] 0,&\widetilde x\leq0. \end{cases} Equivalently, this is the mixture \boxed{ \frac{a_n}{a_n+1}\operatorname{Uniform}(0,b_n) +\frac{1}{a_n+1}\operatorname{Pareto}(a_n,b_n). } The component weights sum to one. The Uniform component has no probability above b_n, whereas the Pareto component lies above b_n almost surely. Therefore, \boxed{ \mathbb{P}(\widetilde X>b_n\mid{\boldsymbol{x}}) =\frac{1}{a_n+1} =\frac{1}{\alpha+n+1}. } This is the posterior predictive probability that the next captured serial exceeds whichever is larger: the confirmed lower bound \beta or the current sample maximum.
(d) Why there is no contradiction with the Pitman–Koopman–Darmois theorem
In a regular exponential-family model, the support cannot change with the parameter: a representation of the form p(x\mid\theta) =b(x)\frac{\exp\{c(\theta)T(x)\}}{a(\theta)} has support determined by b(x), since the exponential term is positive. For the Uniform model, p(x\mid\theta)=\frac{1}{\theta} \mathbf{1}\{0<x<\theta\}, and the indicator switches on or off as \theta changes. Its support depends on \theta, so this is an irregular model rather than a regular exponential family.
The Pitman–Koopman–Darmois theorem assumes, among other regularity conditions, parameter-independent support. This model violates that assumption and lies outside the theorem’s scope. It may therefore have the fixed-dimensional sufficient statistic x_{(n)} and a conjugate prior without being an exponential family. The existence of a conjugate prior does not by itself imply exponential-family structure.
(e) Numerical posterior and 95% HPD interval
Here n=4, x_{(n)}=60, \alpha=1, and \beta=50, so a_n=5, \qquad b_n=60, \qquad \theta\mid{\boldsymbol{x}}\sim\operatorname{Pareto}(5,60). The posterior mean is \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}})=\frac{5(60)}{4}=75. } The posterior density is strictly decreasing on [60,\infty), so its highest-density interval must begin at the posterior mode and lower support boundary, 60. If the upper endpoint is u, then 0.05 =\mathbb{P}(\theta>u\mid{\boldsymbol{x}}) =\left(\frac{60}{u}\right)^5. Hence u=60(20)^{1/5}\approx109.2, and \boxed{ \text{95\% HPD interval} =\left[60,\,60(20)^{1/5}\right] \approx[60,109.2]. } The lower endpoint is 60 because values below the largest observed serial are impossible, and the decreasing posterior density is maximised at that boundary.