Which groups differ?
Section of Analysis of variance examples, part of Data analysis
Which groups differ?
| Type | Section |
| Taught in | Analysis of variance examples |
| Part | Data analysis |
A significant F says the means are not all equal; it does not say which ones differ. Comparing every pair with its own test at level \alpha inflates the chance of a false difference, since with five groups there are ten pairs.
Definition 1 (Bonferroni correction) To make m comparisons with a familywise error rate of at most \alpha, run each at level \alpha/m. Equivalently, multiply each p-value by m, capping at 1.
The correction works because the probability that at least one of m events occurs is at most the sum of their probabilities. It is simple and always valid, but conservative when m is large. Tukey intervals, built for all pairwise differences of means, are sharper. For the cold sores, Tukey’s method at familywise level 0.05 separates only one pair: placebo against treatment 3, with difference 3.17 days and 95% simultaneous interval (0.57,\ 5.76). The Bonferroni version agrees: the t statistic for that pair is 3.59, beyond the corrected critical value t_{25;\,1-0.05/20}=3.08, and no other pair passes.
TukeyHSD(aov(times ~ treatment, data = coldsore))
pairwise.t.test(coldsore$times, coldsore$treatment, p.adjust.method = "bonferroni")The same question arises with unequal groups. Haemoglobin was measured for three forms of sickle-cell disease, in groups of 16 (HbSS), 15 (HbST) and 10 (HbSC). The analysis of variance gives a treatment sum of squares of 99.89 on 2 degrees of freedom and an error mean square of 1.00 on 38, so F=49.9 and the p-value is about 2.3\times10^{-11}. Bonferroni-corrected pairwise tests, with three comparisons each at level 0.05/3, then say which forms differ; the exercises work through them.
Used in
See also
Bonferroni correction · Interaction · Checking an analysis-of-variance model · Two-factor analysis of variance · Randomisation and blocking · Main effect