Regression tree
Topic of Classification trees, part of Statistical learning
Regression tree
| Type | Topic |
| Taught in | Classification trees |
| Part | Statistical learning |
A tree whose leaves predict a numerical response by the mean of the training responses in the leaf. It is grown by choosing at each node the split that most reduces the residual sum of squares, and pruned like a classification tree.
In Classification trees
From Classification trees
The price of the readable partition is the way it is grown. At each node the algorithm takes the split that most improves a local impurity score. That greedy step does not look ahead, so an early split that is only slightly better than its rival can commit the rest of the tree. Pruning, or a penalty on the number of leaves, is what stops the partition from cutting until every training case sits in a pure leaf. Even a pruned tree need not be stable. A small change in the sample can change the first split, and every later branch is then a branch of a different tree. That instability is the reason for the averages on Ensembles. Regression trees assign a number rather than a class to each leaf. The splitting idea is the same, and this page stays with classification.
Used in
See also
Decision tree · Node impurity · Building a classification tree · Choosing a split · Invariance to a change of units · Multi-way splits and missing values · Tree size and pruning · What a fitted tree can be trusted to mean