Fisher–Neyman factorisation
Theorem taught in Exponential families and conjugacy, part of Bayesian statistics
Fisher–Neyman factorisation
| Type | Theorem |
| Taught in | Exponential families and conjugacy |
| Part | Bayesian statistics |
| Worked examples | 2 |
| Questions using it | 2 |
Suppose the family \{p({\boldsymbol{x}}\mid\theta):\theta\in\Omega\} is dominated by a common measure. Then T({\boldsymbol{X}}) is sufficient for \theta if and only if there are nonnegative functions g_\theta and h such that p({\boldsymbol{x}}\mid\theta) =g_\theta\{T({\boldsymbol{x}})\}h({\boldsymbol{x}}), \qquad \theta\in\Omega, where h does not depend on \theta.
Examples
Sufficiency for a normal mean
Worked example from Exponential families and conjugacy
Suppose
X_1,\ldots,X_n\mid\mu \stackrel{\mathrm{iid}}{\sim}\operatorname{N}(\mu,\sigma^2), \qquad \sigma^2\text{ known}.
Question. Use the Fisher–Neyman factorisation theorem to find a sufficient statistic for \mu. Then identify a sufficient statistic when both \mu and \sigma^2 are unknown.
Solution. The joint density is \begin{align*} p({\boldsymbol{x}}\mid\mu) &=(2\pi\sigma^2)^{-n/2} \exp\!\left\{-\frac1{2\sigma^2} \sum_{i=1}^n(x_i-\mu)^2\right\}. \end{align*} Expand the quadratic: \begin{align*} \sum_{i=1}^n(x_i-\mu)^2 &=\sum_{i=1}^n(x_i^2-2\mu x_i+\mu^2)\\ &=\sum_{i=1}^nx_i^2 -2\mu\sum_{i=1}^nx_i +n\mu^2. \end{align*} Substitution gives \begin{align*} p({\boldsymbol{x}}\mid\mu) &=\underbrace{ (2\pi\sigma^2)^{-n/2} \exp\!\left\{-\frac1{2\sigma^2}\sum_{i=1}^nx_i^2\right\}} _{h({\boldsymbol{x}})}\\ &\quad\times \underbrace{ \exp\!\left\{ \frac{\mu}{\sigma^2}\sum_{i=1}^nx_i -\frac{n\mu^2}{2\sigma^2}\right\}} _{g_\mu\{T({\boldsymbol{x}})\}}. \end{align*} All dependence on \mu passes through
T({\boldsymbol{x}})=\sum_{i=1}^nx_i.
The factorisation theorem therefore proves that \sum_iX_i is sufficient for \mu. Since \overline X=T/n is a one-to-one function of T when n is fixed, \overline X is equivalent and is also sufficient.
If both \mu and \sigma^2 are unknown, the displayed factorisation no longer isolates all parameter dependence through \sum_iX_i. A sufficient statistic for (\mu,\sigma^2) is instead
\left(\sum_{i=1}^nX_i,\sum_{i=1}^nX_i^2\right),
or equivalently
\left(\overline X,\sum_{i=1}^n(X_i-\overline X)^2\right).
This is the nuisance-parameter warning in concrete form.
Sufficiency and the posterior
Question page: Sufficiency and the posterior
Let {\boldsymbol{X}}=(X_1,\ldots,X_n)^{\mathsf T} have likelihood p({\boldsymbol{x}}\mid\theta) and prior p(\theta). If U({\boldsymbol{X}}) is sufficient for \theta, use the Fisher–Neyman factorisation theorem and Bayes’ rule to show that the posterior depends on the observed sample only through U({\boldsymbol{x}}). Retain the normalising integral in the derivation.
Question. Let {\boldsymbol{X}}=(X_1,\ldots,X_n)^{\mathsf T} have likelihood p({\boldsymbol{x}}\mid\theta) and prior p(\theta). If U({\boldsymbol{X}}) is sufficient for \theta, use the Fisher–Neyman factorisation theorem and Bayes’ rule to show that the posterior depends on the observed sample only through U({\boldsymbol{x}}). Retain the normalising integral in the derivation.
Solution. By the Fisher–Neyman factorisation theorem, sufficiency of U({\boldsymbol{X}}) implies that there are nonnegative functions g and h such that
p({\boldsymbol{x}}\mid\theta)=g\{U({\boldsymbol{x}}),\theta\}h({\boldsymbol{x}}),
where h does not depend on \theta. Bayes’ rule, including its normalising integral, gives \begin{align*} p(\theta\mid{\boldsymbol{x}}) &=\frac{p({\boldsymbol{x}}\mid\theta)p(\theta)} {\displaystyle\int p({\boldsymbol{x}}\mid t)p(t)\,\mathrm{d} t}\\ &=\frac{g\{U({\boldsymbol{x}}),\theta\}h({\boldsymbol{x}})p(\theta)} {\displaystyle\int g\{U({\boldsymbol{x}}),t\}h({\boldsymbol{x}})p(t)\,\mathrm{d} t}\\ &=\boxed{ \frac{g\{U({\boldsymbol{x}}),\theta\}p(\theta)} {\displaystyle\int g\{U({\boldsymbol{x}}),t\}p(t)\,\mathrm{d} t}}. \end{align*} Every occurrence of the observed sample is through U({\boldsymbol{x}}); the factor h({\boldsymbol{x}}) cancels because it is constant with respect to the integration variable and the parameter.
Used in
- Taught in Exponential families and conjugacy, where the derivation and examples are.
- Questions that use it (2) — Factorisation and data reduction · Sufficiency and the posterior
- All 18 questions on Exponential families and conjugacy
See also
Statistic and estimator · Sufficient statistic · Minimal sufficiency · Complete statistic · Ancillary statistic · Basu’s theorem · Natural exponential family · Translation to another common notation