A step-up subtlety

Topic: High-dimensional analysis · more questions on this topic
‹ A Bonferroni threshold Soft threshold ›

Five sorted p-values are 0.001, 0.025, 0.026, 0.027, 0.800, with q^*=0.05 and m=5. Apply Benjamini–Hochberg: list i q^*/m and find k. Is H_{(2)} rejected even though p_{(2)}=0.025 exceeds its own threshold 2(0.05)/5=0.02? Why does the procedure reject all hypotheses up to k even if some earlier p-values exceed their own thresholds?

Worked answer (click to show)

The thresholds i q^*/m are 0.01, 0.02, 0.03, 0.04, 0.05. Comparing: p_{(1)}=0.001\le 0.01, yes; p_{(2)}=0.025>0.02, no; p_{(3)}=0.026\le 0.03, yes; p_{(4)}=0.027\le 0.04, yes; p_{(5)}=0.800>0.05, no. The largest i meeting the condition is k=4, so H_{(1)},\ldots,H_{(4)} are all rejected, including H_{(2)}, even though its p-value exceeds its own threshold.

The procedure is step-up. Working up from the largest p-value, the first one below its line triggers rejection of all smaller p-values. If the data support rejecting the hypothesis with the fourth-smallest p-value at the stated level, then the hypotheses with smaller p-values are stronger evidence and are rejected with it. Declaring p_{(4)} significant and p_{(2)} not would be incoherent for this rule, because the rule uses one selected cutoff.