Bootstrap error and the correction
Definition taught in Estimating error rates, part of Statistical learning
Bootstrap error and the correction
| Type | Definition |
| Taught in | Estimating error rates |
| Part | Statistical learning |
Efron’s bootstrap estimator scores each observation only on the bootstrap samples that excluded it. If C_{-i} is the set of bootstrap samples that do not contain observation i, and \eta_b is the classifier trained on bootstrap sample b,
\widehat{\mathrm{Err}}^{(\mathrm{BOOT})} =\frac1n\sum_{i=1}^n \frac{1}{|C_{-i}|} \sum_{b\in C_{-i}} Q\bigl(y_i,\eta_b(x_i)\bigr).
Each case is tested only by rules that did not see it, the same discipline as cross-validation. Because each bootstrap training set contains only about 0.632n distinct observations, the trained rules are weaker than a rule trained on all n distinct cases, and \widehat{\mathrm{Err}}^{(\mathrm{BOOT})} is generally biased upward.
The .632 estimator pulls that upward bias back toward the apparent error, with weights taken from the limiting inclusion probability and its complement:
\widehat{\mathrm{Err}}^{(.632)} =0.368\,\mathrm{err}+0.632\,\widehat{\mathrm{Err}}^{(\mathrm{BOOT})}.
Used in
- Taught in Estimating error rates, where the derivation and examples are.
- All 7 questions on Estimating error rates
See also
Estimator · Bootstrap · Cross-validation · Training set · Apparent error · Fixed-rule error · Confusion matrix · Leave-one-out cross-validation