Q12. Pareto likelihood and Gamma posterior
Suppose X_1,\ldots,X_n are conditionally iid with density p(x_i\mid\theta) =\theta m^\theta x_i^{-(\theta+1)}, \qquad x_i\geq m, where m>0 is known and \theta>0 is unknown. Use the prior p(\theta)=\theta e^{-\theta}, \qquad \theta>0. You may use the fact that a \operatorname{Gamma}(A,B) distribution in shape–rate form has kernel \theta^{A-1}e^{-B\theta}, mean A/B, and mode (A-1)/B when A>1.
[8 marks]
(a) Show that the likelihood can be written L(\theta;{\boldsymbol{x}})\propto\theta^n e^{-\theta S}, \qquad S=\sum_{i=1}^n\log\left(\frac{x_i}{m}\right). [3 marks]
(b) Derive the posterior distribution of \theta. Identify its distribution, parameters, and support. [3 marks]
(c) State the posterior mean and MAP estimate. [2 marks]
Start with part (a). The target expression is supplied, so the task is to show how multiplying the densities produces it.
Definitions. The known lower endpoint is m>0, and the observed data satisfy x_i\ge m. We use the shape–rate Gamma kernel \theta^{A-1}e^{-B\theta} for \theta>0, A,B>0, with mean A/B and mode (A-1)/B when A>1. The MAP estimate is this posterior mode. The prior \theta e^{-\theta} is \operatorname{Gamma}(2,1).
Since m is known and the observed sample is valid, the support indicator does not depend on \theta.
(a) Likelihood
By conditional independence, \begin{aligned} L(\theta;{\boldsymbol{x}}) &=\prod_{i=1}^n \left[\theta m^\theta x_i^{-(\theta+1)}\right]\\ &=\theta^n m^{n\theta} \prod_{i=1}^n x_i^{-(\theta+1)}. \end{aligned} The factor m^{n\theta} must be retained because it depends on \theta. Using a^b=e^{b\log a}, \begin{aligned} L(\theta;{\boldsymbol{x}}) &=\theta^n \exp\left(n\theta\log m\right) \exp\left[-(\theta+1)\sum_{i=1}^n\log x_i\right]\\ &=\theta^n \exp\left[-\sum_{i=1}^n\log x_i\right] \exp\left[-\theta\left\{ \sum_{i=1}^n\log x_i-n\log m \right\}\right]. \end{aligned} The first exponential in the final line does not depend on \theta and may be absorbed into the proportionality constant. Also, \sum_{i=1}^n\log x_i-n\log m =\sum_{i=1}^n\log\left(\frac{x_i}{m}\right) =S. Therefore, \boxed{ L(\theta;{\boldsymbol{x}})\propto\theta^n e^{-\theta S}, \qquad \theta>0. }
(b) Posterior distribution
Multiplying the likelihood by the prior gives \begin{aligned} p(\theta\mid{\boldsymbol{x}}) &\propto L(\theta;{\boldsymbol{x}})p(\theta)\\ &\propto \theta^n e^{-\theta S}\, \theta e^{-\theta}\\ &=\theta^{n+1}e^{-(S+1)\theta}, \qquad \theta>0. \end{aligned} Comparing this with the shape–rate Gamma kernel \theta^{A-1}e^{-B\theta}, we obtain A-1=n+1, \qquad B=S+1. Hence \boxed{ \theta\mid{\boldsymbol{x}} \sim\operatorname{Gamma}(n+2,S+1) \quad\text{in shape--rate form}, \qquad \theta>0. } Because x_i\geq m, each \log(x_i/m)\geq0, so S+1>0 as required.
(c) Posterior mean and MAP estimate
For a shape–rate \operatorname{Gamma}(A,B) distribution, the mean is A/B. Therefore, \boxed{ \mathbb{E}(\theta\mid{\boldsymbol{x}})=\frac{n+2}{S+1}. } The MAP estimate is the posterior mode. Since A=n+2>1, \boxed{ \widehat{\theta}_{\mathrm{MAP}} =\frac{A-1}{B} =\frac{n+1}{S+1}. }