Mean and variance of a mixture

For the mixture with component means \mu_h and variances \sigma_h^2 in one dimension, show that \mathbb{E} X=\sum_h\pi_h\mu_h=:\bar\mu and

\operatorname{Var} X=\sum_h\pi_h\sigma_h^2+\sum_h\pi_h(\mu_h-\bar\mu)^2.

Evaluate for \pi=(0.3,0.7), \mu=(0,4), \sigma^2=(1,4).

Condition on the component label. There is variation inside each component and variation between its mean and the overall mean. Both contribute.

\mathbb{E} X=\sum_h\pi_h\mu_h.

For the variance use \mathbb{E} X^2=\sum_h\pi_h(\sigma_h^2+\mu_h^2), so

\operatorname{Var} X =\sum_h\pi_h\sigma_h^2+\sum_h\pi_h\mu_h^2-\bar\mu^2 =\sum_h\pi_h\sigma_h^2+\sum_h\pi_h(\mu_h-\bar\mu)^2.

The last step expands \sum_h\pi_h(\mu_h-\bar\mu)^2=\sum_h\pi_h\mu_h^2-\bar\mu^2. This is the law of total variance with the component label as the conditioning variable: within-component variance plus between-component variance.

Numerically \bar\mu=0.3\cdot 0+0.7\cdot 4=2.8 and

\operatorname{Var} X =(0.3\cdot 1+0.7\cdot 4)+0.3\cdot 2.8^2+0.7\cdot 1.2^2 =3.1+(2.352+1.008) =6.46.