Exponential observations parameterised by mean
Topic: Exponential families and conjugacy · more questions on this topic
‹ Exponential observations parameterised by rate Q1 ›
Let X_1,\ldots,X_n be iid \operatorname{Exp}(1/\theta) observations, so that \theta>0 is the mean and
p(x_i\mid\theta)=\theta^{-1}e^{-x_i/\theta}\mathbf{1}_{\{x_i\geq0\}}.
Write one observation in the form
p(x\mid\theta)=\frac{b(x)\exp\{c(\theta)T(x)\}}{a(\theta)}
and use the conjugate prior kernel
p(\theta\mid\alpha,\beta)\propto\frac{\exp\{c(\theta)\alpha\}}{a(\theta)^\beta}
to find a conjugate prior and derive its posterior parameters.
Question. Let X_1,\ldots,X_n be iid \operatorname{Exp}(1/\theta) observations, so that \theta>0 is the mean and
p(x_i\mid\theta)=\theta^{-1}e^{-x_i/\theta}\mathbf{1}_{\{x_i\geq0\}}.
Write one observation in the form p(x\mid\theta)=b(x)\exp\{c(\theta)T(x)\}/a(\theta) and use the conjugate prior kernel p(\theta\mid\alpha,\beta)\propto\exp\{c(\theta)\alpha\}/a(\theta)^\beta to find a conjugate prior and derive its posterior parameters.
Solution. Now
p(x\mid\theta)=\theta^{-1}e^{-x/\theta}\mathbf{1}_{\{x\geq0\}} =\frac{b(x)\exp\{c(\theta)T(x)\}}{a(\theta)}
with
\boxed{ b(x)=\mathbf{1}_{\{x\geq0\}},\quad T(x)=x,\quad c(\theta)=-\frac1\theta,\quad a(\theta)=\theta}.
The generic prior kernel becomes \begin{align*} p(\theta\mid\alpha,\beta) &\propto \frac{\exp(-\alpha/\theta)}{\theta^\beta}\\ &=\theta^{-\beta}e^{-\alpha/\theta}. \end{align*} Since an \operatorname{InvGamma}(a,b) density has kernel \theta^{-a-1}e^{-b/\theta}, this prior is
\theta\sim\operatorname{InvGamma}(\beta-1,\alpha), \qquad \alpha>0,\quad\beta>1.
The sample likelihood is
p({\boldsymbol{x}}\mid\theta) \propto\theta^{-n} \exp\!\left[-\frac{S({\boldsymbol{x}})}{\theta}\right].
Therefore \begin{align*} p(\theta\mid{\boldsymbol{x}}) &\propto \theta^{-n}e^{-S({\boldsymbol{x}})/\theta} \theta^{-\beta}e^{-\alpha/\theta}\\ &=\theta^{-(\beta+n)} \exp\!\left[-\frac{\alpha+S({\boldsymbol{x}})}{\theta}\right], \end{align*} and hence
\boxed{\theta\mid{\boldsymbol{x}} \sim\operatorname{InvGamma}\bigl(\beta+n-1,\alpha+S({\boldsymbol{x}})\bigr)}.
Equivalently, a standard prior \operatorname{InvGamma}(a_0,b_0) updates to
\theta\mid{\boldsymbol{x}}\sim \operatorname{InvGamma}\bigl(a_0+n,b_0+S({\boldsymbol{x}})\bigr).
\operatorname{Exp}(\theta) in the preceding exercise used \theta as a rate, whereas \operatorname{Exp}(1/\theta) here makes \theta the mean. The first likelihood is conjugate to a Gamma prior for \theta; the second is conjugate to an inverse-Gamma prior. Interchanging rate and mean without rewriting the density would interchange the two answers.