Building a classification tree
Section of Classification trees, part of Statistical learning
Building a classification tree
| Type | Section |
| Taught in | Classification trees |
| Part | Statistical learning |
At a node, compare possible splits and choose one that makes the children purer according to a stated criterion. Repeat inside each child. This is a greedy search: a split is judged by its immediate improvement, not by every tree that could eventually grow beneath it. Stopping rules and pruning control the size. Contradictory labels at identical predictor values cannot be separated by further predictor-based splits.
After the tree is built, each training case is passed from the root along the branch its measurements select, until it reaches a leaf. The class proportions in that leaf are recorded, and the leaf is labelled with its majority class. A new x follows the same questions and receives the label of the leaf it reaches. The proportions in the leaf are a crude local estimate of the posterior \tau_g(x), constant on the rectangle the leaf owns. The Bayes rule would use the true posterior. The tree uses the majority in the rectangle that contains x.
The geometry is a partition of the predictor space by thresholds on single coordinates. The same partition reads as a nested list of rules. Four questions organise the rest of the page: which impurity a split should reduce, how the best split of a given variable is found, when splitting stops, and what a fitted tree can and cannot be trusted to mean.
Used in
See also
Node impurity · Choosing a split · Invariance to a change of units · Multi-way splits and missing values · Tree size and pruning · What a fitted tree can be trusted to mean · Laboratory: prune using training cross-validation · Decision tree