Q49. A Gibbs full conditional under independent priors
[6 marks]
Suppose X_i\mid\mu,\tau\overset{\mathrm{iid}}{\sim} N\!\left(\mu,\frac1\tau\right),\qquad i=1,\ldots,n, where \mu\in\mathbb R and \tau>0 are unknown.
Assign independent priors \mu\sim N\!\left(\mu_0,\frac1{\kappa_0}\right), \qquad \tau\sim\operatorname{Gamma}(a_0,b_0), where all hyperparameters are known and \kappa_0,a_0,b_0>0. The Gamma distribution is parametrised by shape and rate.
(a) Derive the full conditional distribution \tau\mid\mu,{\boldsymbol{x}}. Show its kernel and identify the distribution and parameters. [4 marks]
(b) Does this full conditional depend explicitly on \mu_0 or \kappa_0? Explain why or why not. [2 marks]
Definitions. Precision is \tau=1/\sigma^2. N(m,v) uses variance v, and \operatorname{Gamma}(a,b) uses shape a and rate b, with kernel \tau^{a-1}e^{-b\tau} on \tau>0. The full conditional for \tau holds \mu and the data fixed, so only factors that vary with \tau remain.
Assumptions. The priors are independent: \mu\sim N(\mu_0,1/\kappa_0) and \tau\sim\operatorname{Gamma}(a_0,b_0), with \kappa_0,a_0,b_0>0. In particular, the prior variance of \mu does not depend on \tau; this distinguishes the question from a Normal–Gamma conditional prior.
(a) Full conditional for \tau. The Normal density with precision \tau is f(x_i\mid\mu,\tau) =\left(\frac\tau{2\pi}\right)^{1/2} \exp\left\{-\frac\tau2(x_i-\mu)^2\right\}. Therefore the likelihood kernel is L(\mu,\tau;{\boldsymbol{x}})\propto \tau^{n/2}\exp\left\{-\frac\tau2\sum_{i=1}^n(x_i-\mu)^2\right\}. The independent priors give the joint posterior kernel \begin{aligned} p(\mu,\tau\mid{\boldsymbol{x}}) &\propto \tau^{n/2}\exp\left\{-\frac\tau2\sum_i(x_i-\mu)^2\right\}\\ &\quad\times\exp\left\{-\frac{\kappa_0}{2}(\mu-\mu_0)^2\right\} \tau^{a_0-1}e^{-b_0\tau}. \end{aligned} For fixed \mu, its prior factor is constant with respect to \tau. Dropping that factor and collecting powers and exponents gives \boxed{ p(\tau\mid\mu,{\boldsymbol{x}})\propto \tau^{a_0+n/2-1} \exp\left[-\left\{b_0+\frac12\sum_i(x_i-\mu)^2\right\}\tau\right], \quad\tau>0. } Hence \boxed{ \tau\mid\mu,{\boldsymbol{x}}\sim\operatorname{Gamma}\!\left( a_0+\frac n2,\ b_0+\frac12\sum_{i=1}^n(x_i-\mu)^2\right) \quad\text{(shape--rate)}. }
(b) Dependence on \mu_0 and \kappa_0. No: the full conditional has no explicit dependence on these two hyperparameters. Under the independent priors, p(\mu,\tau)=p(\mu)p(\tau), and once \mu is conditioned on, p(\mu) is a positive constant in the conditional kernel for \tau. It cancels on normalisation.
This does not mean that the prior for \mu is irrelevant to all inference about \tau. It affects the posterior distribution of \mu, and thus can affect the marginal posterior of \tau after \mu is integrated out.
Key result. Do not silently replace the independent prior with \mu\mid\tau\sim N(\mu_0,1/(\kappa_0\tau)). That is a different model whose prior contributes additional \tau-dependent factors to the full conditional.