Question 2. Normal variance, known mean

Topic: Conjugate analysis for normal models · more questions on this topic
‹ Question 1 Question 3 ›

Let

X_i\mid\sigma^2\stackrel{\mathrm{iid}}{\sim}\operatorname{N}(\mu,\sigma^2),

with \mu known and

\sigma^2\sim\operatorname{InvGamma}(\alpha_0,\beta_0),

in the shape–rate parameterisation

f(\sigma^2)\propto(\sigma^2)^{-\alpha_0-1}\exp(-\beta_0/\sigma^2).

(a) Write the posterior distribution of \sigma^2, and state what is added to each parameter.

(b) State the posterior mean and mode of \sigma^2, including the conditions under which each exists.

(c) Explain why that mean and mode generally differ.

(d) Explain why the variance update uses squared deviations from the known mean \mu, rather than from \bar x.

Worked answer (click to show)

Definitions. Put v=\sigma^2 and Q_\mu=\sum_i(x_i-\mu)^2, with \mu known. Our \operatorname{InvGamma}(a,b) convention has density b^a v^{-a-1}e^{-b/v}/\Gamma(a) for v>0. Equivalently, 1/v\sim\operatorname{Gamma}(a,b) in shape–rate form. Specifying the kernel avoids ambiguity about the second parameter.

(a) Independence gives the likelihood kernel L(v)\propto v^{-n/2}e^{-Q_\mu/(2v)}. Multiplication by the prior gives

p(v\mid{\boldsymbol{x}})\propto v^{-n/2}e^{-Q_\mu/(2v)}v^{-\alpha_0-1}e^{-\beta_0/v} =v^{-(\alpha_0+n/2)-1}e^{-(\beta_0+Q_\mu/2)/v}.

Let \alpha_n=\alpha_0+n/2 and \beta_n=\beta_0+Q_\mu/2. Recognising this kernel,

\boxed{ \sigma^2\mid{\boldsymbol{x}} \sim \operatorname{InvGamma}(\alpha_n,\beta_n). }

Each observation adds 1/2 to the shape. Half the sum of squared deviations from the known mean is added to the rate.

(b) For this shape–rate kernel, and for \beta_n>0,

\mathbb{E}(\sigma^2\mid{\boldsymbol{x}})=\frac{\beta_n}{\alpha_n-1} \quad\text{when }\alpha_n>1, \qquad \operatorname{mode}(\sigma^2\mid{\boldsymbol{x}})=\frac{\beta_n}{\alpha_n+1} \quad\text{when }\alpha_n>0.

The mode solves (-\alpha_n-1)/v+\beta_n/v^2=0. The mean is the gamma integral of v\cdot v^{-\alpha_n-1}e^{-\beta_n/v}, which converges only for \alpha_n>1.

More explicitly, substituting u=\beta_n/v into the expectation gives \mathbb{E}(v\mid{\boldsymbol{x}})=\beta_n\Gamma(\alpha_n-1)/\Gamma(\alpha_n)=\beta_n/(\alpha_n-1). When 0<\alpha_n\le1, the posterior is still a proper distribution, but its right tail is too heavy to have a finite mean. Posterior propriety and existence of a posterior moment are separate requirements.

(c) For \alpha_n>1,

\frac{\beta_n}{\alpha_n-1}-\frac{\beta_n}{\alpha_n+1} = \frac{2\beta_n}{(\alpha_n-1)(\alpha_n+1)}>0.

The inverse-gamma density is right-skewed, so the long right tail pulls the mean above the mode.

(d) Under the sampling model, \operatorname{Var}(X_i\mid\sigma^2)=\mathbb{E}[(X_i-\mu)^2\mid\sigma^2]. The sufficient statistic for \sigma^2, with \mu known, is \sum(x_i-\mu)^2. Since

\sum_{i=1}^n(x_i-\bar x)^2 = \sum_{i=1}^n(x_i-\mu)^2-n(\bar x-\mu)^2,

centring at \bar x omits the term n(\bar x-\mu)^2.

When the mean is known, departures of the sample mean from that value are evidence about the observation variance too. For example, a tightly clustered sample far from the known mean has small \sum_i(x_i-\bar x)^2 but large Q_\mu. Replacing Q_\mu by the centred sum of squares would discard that evidence and change the likelihood.