Posterior mean under squared-error loss
For observed data {\boldsymbol{x}}, consider the posterior mean squared error of a point estimate d=d({\boldsymbol{x}}),
R(d\mid{\boldsymbol{x}})=\mathbb{E}\bigl[(d-\theta)^2\mid{\boldsymbol{x}}\bigr].
Show, without omitting the quadratic expansion or the minimisation step, that the posterior mean d=\mathbb{E}(\theta\mid{\boldsymbol{x}}) is the minimum-mean-square-error estimator of \theta.
Question. For observed data {\boldsymbol{x}}, consider the posterior mean squared error of a point estimate d=d({\boldsymbol{x}}),
R(d\mid{\boldsymbol{x}})=\mathbb{E}\bigl[(d-\theta)^2\mid{\boldsymbol{x}}\bigr].
Show, without omitting the quadratic expansion or the minimisation step, that the posterior mean d=\mathbb{E}(\theta\mid{\boldsymbol{x}}) is the minimum-mean-square-error estimator of \theta.
Solution. Once {\boldsymbol{x}} has been observed, d=d({\boldsymbol{x}}) is a fixed number and only \theta is random under the posterior. Therefore \begin{align*} R(d\mid{\boldsymbol{x}}) &=\mathbb{E}\bigl[(d-\theta)^2\mid{\boldsymbol{x}}\bigr]\\ &=\mathbb{E}\bigl[d^2-2d\theta+\theta^2\mid{\boldsymbol{x}}\bigr]\\ &=d^2-2d\mathbb{E}(\theta\mid{\boldsymbol{x}})+\mathbb{E}(\theta^2\mid{\boldsymbol{x}}). \end{align*} Put m=\mathbb{E}(\theta\mid{\boldsymbol{x}}). Adding and subtracting m^2 gives \begin{align*} R(d\mid{\boldsymbol{x}}) &=d^2-2dm+m^2+\mathbb{E}(\theta^2\mid{\boldsymbol{x}})-m^2\\ &=(d-m)^2+\operatorname{Var}(\theta\mid{\boldsymbol{x}}). \end{align*} The posterior variance does not depend on d, and (d-m)^2\geq0 with equality exactly when d=m. Equivalently,
\frac{\partial R}{\partial d}=2d-2m, \qquad \frac{\partial^2 R}{\partial d^2}=2>0.
Thus the unique minimiser is
\boxed{d_{\mathrm{MMSE}}({\boldsymbol{x}})=\mathbb{E}(\theta\mid{\boldsymbol{x}})}.