Sufficiency and the posterior

Topic: Exponential families and conjugacy · more questions on this topic
‹ Conjugacy is not guaranteed by a name Exponential observations parameterised by rate ›

Let {\boldsymbol{X}}=(X_1,\ldots,X_n)^{\mathsf T} have likelihood p({\boldsymbol{x}}\mid\theta) and prior p(\theta). If U({\boldsymbol{X}}) is sufficient for \theta, use the Fisher–Neyman factorisation theorem and Bayes’ rule to show that the posterior depends on the observed sample only through U({\boldsymbol{x}}). Retain the normalising integral in the derivation.

Worked answer (click to show)

Question. Let {\boldsymbol{X}}=(X_1,\ldots,X_n)^{\mathsf T} have likelihood p({\boldsymbol{x}}\mid\theta) and prior p(\theta). If U({\boldsymbol{X}}) is sufficient for \theta, use the Fisher–Neyman factorisation theorem and Bayes’ rule to show that the posterior depends on the observed sample only through U({\boldsymbol{x}}). Retain the normalising integral in the derivation.

Solution. By the Fisher–Neyman factorisation theorem, sufficiency of U({\boldsymbol{X}}) implies that there are nonnegative functions g and h such that

p({\boldsymbol{x}}\mid\theta)=g\{U({\boldsymbol{x}}),\theta\}h({\boldsymbol{x}}),

where h does not depend on \theta. Bayes’ rule, including its normalising integral, gives \begin{align*} p(\theta\mid{\boldsymbol{x}}) &=\frac{p({\boldsymbol{x}}\mid\theta)p(\theta)} {\displaystyle\int p({\boldsymbol{x}}\mid t)p(t)\,\mathrm{d} t}\\ &=\frac{g\{U({\boldsymbol{x}}),\theta\}h({\boldsymbol{x}})p(\theta)} {\displaystyle\int g\{U({\boldsymbol{x}}),t\}h({\boldsymbol{x}})p(t)\,\mathrm{d} t}\\ &=\boxed{ \frac{g\{U({\boldsymbol{x}}),\theta\}p(\theta)} {\displaystyle\int g\{U({\boldsymbol{x}}),t\}p(t)\,\mathrm{d} t}}. \end{align*} Every occurrence of the observed sample is through U({\boldsymbol{x}}); the factor h({\boldsymbol{x}}) cancels because it is constant with respect to the integration variable and the parameter.