Behrens–Fisher problem
Topic of Estimation and inference for the multivariate normal, part of Statistical learning · Also in Estimation examples
Behrens–Fisher problem
| Type | Topic |
| Taught in | Estimation and inference for the multivariate normal · Estimation examples |
| Part | Statistical learning · Data analysis |
Comparing two normal means when the two variances are unknown and not assumed equal. There is no exact pivot; Welch’s approximation is the usual univariate answer, and the same difficulty returns for two mean vectors with unequal covariance matrices.
In Estimation and inference for the multivariate normal
Pooling is justified by the common-covariance model. Equal group sizes do not make that assumption true, and they do not restore the exact finite-sample F distribution under arbitrary heteroscedasticity. Large-sample robustness of the pooled procedure is a more limited conclusion. With unequal covariances the problem is the multivariate Behrens–Fisher problem, even when the group sizes happen to match.
In Estimation examples
From Estimation examples
The pooled interval for \mu_X-\mu_Y assumes the two populations have the same variance. Without that assumption, \overline X-\overline Y is still approximately normal with variance \sigma_X^2/n_1+\sigma_Y^2/n_2, and replacing each variance by its sample estimate gives a statistic close to a t distribution with fractional degrees of freedom.
Definition 1 (Welch interval) For independent normal samples of sizes n_1 and n_2 with sample variances s_X^2 and s_Y^2, the Welch interval for \mu_X-\mu_Y with approximate confidence 1-\alpha is
\overline x-\overline y\pm t_{\mathrm{df};\,1-\alpha/2}\sqrt{\frac{s_X^2}{n_1}+\frac{s_Y^2}{n_2}}, \qquad \mathrm{df}=\frac{\bigl(s_X^2/n_1+s_Y^2/n_2\bigr)^2}{\dfrac{(s_X^2/n_1)^2}{n_1-1}+\dfrac{(s_Y^2/n_2)^2}{n_2-1}}.
The degrees of freedom are the Welch–Satterthwaite approximation.
The effective degrees of freedom lie between the smaller of n_1-1 and n_2-1 and the pooled n_1+n_2-2. For non-normal data the interval is still approximately right when both samples are reasonably large, say 30 or more; for small, skewed samples a rank-based method such as the Wilcoxon rank-sum test is safer.
Example 1 (First- and third-year weights) 15 first-year students have mean weight 62.0 kg and standard deviation 15 kg; 10 third-year students have mean 71.5 kg and standard deviation 12 kg.
Question. Give a 95% interval for the difference in mean weights without assuming equal variances.
Solution. The standard error is \sqrt{15^2/15+12^2/10}=5.422, and the effective degrees of freedom are 22.10, so t_{22.10;0.975}=2.073. The interval is
-9.5\pm2.073\times5.422=(-20.74,\ 1.74).
It contains 0, so the data do not show a difference in mean weight. The pooled interval, which assumes equal variances, is (-21.24,\ 2.24) on 23 degrees of freedom, a little wider here because the larger sample has the larger variance. With the raw data, t.test(x, y) gives the Welch interval, R’s default, and t.test(x, y, var.equal = TRUE) the pooled one.
Used in
See also
Welch interval · Variance · Pivot · Mean vector and covariance matrix · Invariance of the maximum likelihood estimator · Wishart distribution · Sampling law of the normal mean and scatter · Multivariate central limit theorem