Exponential observations parameterised by rate

Topic: Exponential families and conjugacy · more questions on this topic
‹ Sufficiency and the posterior Exponential observations parameterised by mean ›

For X\mid\theta\sim\operatorname{Exp}(\theta), take \theta to be the rate, so that

p(x\mid\theta)=\theta e^{-\theta x}\mathbf{1}_{\{x\geq0\}}.

Write the sampling density in the form

p(x\mid\theta)=\frac{b(x)\exp\{c(\theta)T(x)\}}{a(\theta)},

identify a(\theta), b(x), c(\theta) and T(x), and use the conjugate prior kernel

p(\theta\mid\alpha,\beta)\propto\frac{\exp\{c(\theta)\alpha\}}{a(\theta)^\beta}

to find a conjugate prior. Then give the posterior for an iid sample x_1,\ldots,x_n.

Worked answer (click to show)

Question. For X\mid\theta\sim\operatorname{Exp}(\theta), take \theta to be the rate, so that

p(x\mid\theta)=\theta e^{-\theta x}\mathbf{1}_{\{x\geq0\}}.

Write the sampling density in the form p(x\mid\theta)=b(x)\exp\{c(\theta)T(x)\}/a(\theta), identify a(\theta), b(x), c(\theta) and T(x), and use the conjugate prior kernel p(\theta\mid\alpha,\beta)\propto\exp\{c(\theta)\alpha\}/a(\theta)^\beta to find a conjugate prior. Then give the posterior for an iid sample x_1,\ldots,x_n.

Solution. For one observation,

p(x\mid\theta)=\theta e^{-\theta x}\mathbf{1}_{\{x\geq0\}} =\frac{b(x)\exp\{c(\theta)T(x)\}}{a(\theta)}

with

\boxed{ b(x)=\mathbf{1}_{\{x\geq0\}},\quad T(x)=x,\quad c(\theta)=-\theta,\quad a(\theta)=\frac1\theta}.

The conjugate prior kernel for this form is therefore \begin{align*} p(\theta\mid\alpha,\beta) &\propto \frac{\exp\{c(\theta)\alpha\}}{a(\theta)^\beta}\\ &=\frac{e^{-\alpha\theta}}{(1/\theta)^\beta}\\ &=\theta^\beta e^{-\alpha\theta}. \end{align*} For \alpha>0 and \beta>-1, this is

\theta\sim\operatorname{Gamma}(\beta+1,\alpha)

in the shape–rate convention: shape \beta+1 and rate \alpha.

For an iid sample let S({\boldsymbol{x}})=\sum_{i=1}^n x_i. Then \begin{align*} p(\theta\mid{\boldsymbol{x}}) &\propto \left\{\prod_{i=1}^n\theta e^{-\theta x_i}\right\} \theta^\beta e^{-\alpha\theta}\\ &=\theta^n e^{-\theta S({\boldsymbol{x}})} \theta^\beta e^{-\alpha\theta}\\ &=\theta^{\beta+n} e^{-\{\alpha+S({\boldsymbol{x}})\}\theta}. \end{align*} Thus

\boxed{\theta\mid{\boldsymbol{x}} \sim\operatorname{Gamma}\bigl(\beta+n+1,\alpha+S({\boldsymbol{x}})\bigr)}.

Equivalently, if the prior is written in the standard shape–rate form \operatorname{Gamma}(a_0,b_0), then

\theta\mid{\boldsymbol{x}}\sim \operatorname{Gamma}\bigl(a_0+n,b_0+S({\boldsymbol{x}})\bigr).