Random forest
Definition taught in Ensembles, random forests, and boosting, part of Statistical learning
Random forest
| Type | Definition |
| Taught in | Ensembles, random forests, and boosting |
| Part | Statistical learning |
| Questions using it | 2 |
For t=1,\ldots,T, build a forest of T trees as follows.
Definition
- Generate a bootstrap sample (X^{(t)},y^{(t)}) from the training data.
- Fit a tree f_t to (X^{(t)},y^{(t)}). At each node: (a) randomly select k explanatory variables; (b) determine the best explanatory variable and split among them; (c) add this split to the tree. Repeat (a)–(c) until a stopping criterion is met.
For a new observation x the forest predicts, for regression, f(x)=\frac1T\sum_t f_t(x), and for classification
f(x)=\operatorname{arg\,max}_g\sum_{t=1}^T I\bigl(f_t(x)=g\bigr),
or the soft aggregate above if each tree produces class probabilities.
Used in
- Taught in Ensembles, random forests, and boosting, where the derivation and examples are.
- Questions that use it (2) — A tempting wrong turn · Why an ensemble might improve a tree
- All 7 questions on Ensembles, random forests, and boosting
See also
Bootstrap · Training set · Explanatory variable · Ensemble method · Variable importance · Squared error of an average · AdaBoost · The reweighted error is one half