Q1. Rayleigh conjugate prior

Topic: Exponential families and conjugacy · more questions on this topic
‹ Exponential observations parameterised by mean Q2 ›

Suppose X_1,\ldots,X_n\mid\theta \overset{\mathrm{iid}}{\sim}\operatorname{Rayleigh}(\theta), with density f(x\mid\theta) =\frac{x}{\theta}\exp\left(-\frac{x^2}{2\theta}\right), \qquad x>0,\quad \theta>0.

Using the general exponential-family conjugacy result, find a conjugate prior for \theta and derive the posterior distribution.

Worked answer (click to show)

Definitions. For observed {\boldsymbol{x}}=(x_1,\ldots,x_n), L(\theta;{\boldsymbol{x}}) is the sampling density viewed as a function of \theta. A kernel omits only factors constant in that parameter. Conjugacy means that prior and posterior belong to the same family. We use \operatorname{InvGamma}(a,b):\quad p(\theta)\propto\theta^{-a-1}e^{-b/\theta}, \qquad \theta>0,\quad a,b>0, with shape a and inverse-Gamma scale b. The Rayleigh observations are conditionally iid and positive.

The general exponential-family form used here is f(x\mid\theta) =b(x)\frac{\exp\{c(\theta)T(x)\}}{a(\theta)}, for which a conjugate prior has kernel \pi(\theta) \propto \frac{\exp\{\alpha c(\theta)\}}{a(\theta)^\beta}.

For the Rayleigh density, f(x\mid\theta) =x\, \frac{\exp\left\{-\dfrac{1}{\theta}\dfrac{x^2}{2}\right\}} {\theta}. Thus, we identify b(x)=x, \qquad T(x)=\frac{x^2}{2}, \qquad c(\theta)=-\frac{1}{\theta}, \qquad a(\theta)=\theta.

Substitution into the conjugate-prior kernel gives \pi(\theta) \propto \frac{\exp(-\alpha/\theta)}{\theta^\beta} =\theta^{-\beta}\exp\left(-\frac{\alpha}{\theta}\right). Matching this to the inverse-Gamma kernel in Definitions gives a=\beta-1, \qquad b=\alpha. Therefore, a conjugate prior is \boxed{\theta\sim\operatorname{InvGamma}(\beta-1,\alpha)}, with \alpha>0 and \beta>1 for a proper prior.

For an iid sample, the likelihood is \begin{aligned} L(\theta;{\boldsymbol{x}}) &=\prod_{i=1}^n \frac{x_i}{\theta} \exp\left(-\frac{x_i^2}{2\theta}\right)\\ &\propto \theta^{-n} \exp\left(-\frac{1}{2\theta}\sum_{i=1}^n x_i^2\right). \end{aligned} Multiplying the likelihood by the prior gives \begin{aligned} \pi(\theta\mid{\boldsymbol{x}}) &\propto L(\theta;{\boldsymbol{x}})\pi(\theta)\\ &\propto \theta^{-n} \exp\left(-\frac{\sum_{i=1}^n x_i^2}{2\theta}\right) \theta^{-\beta} \exp\left(-\frac{\alpha}{\theta}\right)\\ &\propto \theta^{-(\beta+n)} \exp\left[ -\frac{\alpha+\frac12\sum_{i=1}^n x_i^2}{\theta} \right]. \end{aligned} Recognising the inverse-Gamma kernel, \boxed{ \theta\mid{\boldsymbol{x}} \sim \operatorname{InvGamma}\left( \beta+n-1, \alpha+\frac12\sum_{i=1}^n x_i^2 \right) }.

Equivalently, if the prior is written directly as \theta\sim\operatorname{InvGamma}(a,b), then

Key result. \boxed{ \theta\mid{\boldsymbol{x}} \sim \operatorname{InvGamma}\left( a+n, b+\frac12\sum_{i=1}^n x_i^2 \right) }. This is the form worth remembering after working through the derivation.