A tree by hand

Topic: Classification trees · more questions on this topic
‹ Which split is better? Affine invariance ›

Class 1 has observations x=2,3,10 and class 2 has x=4,5,6,12. Find the first split of a tree using the Gini index. Compute \Delta I for all six cut-points.

Worked answer (click to show)

The ordered data, with class in parentheses, are 2(1), 3(1), 4(2), 5(2), 6(2), 10(1), 12(2). The parent has 3 observations of class 1 and 4 of class 2, so

I_G=2\cdot\frac37\cdot\frac47=\frac{24}{49}=0.490.

Only cuts between consecutive distinct observed values need checking. The reductions are 0.109 at 2.5, 0.261 at 3.5, 0.085 at 4.5, 0.014 at 5.5, 0.004 at 8, and 0.061 at 11. The cut at 3.5 puts the pure class-1 group \{2,3\} on the left. The right child has one class-1 observation and four class-2 observations, so its Gini index is 2(1/5)(4/5)=0.32, and

\Delta I=0.490-\frac57(0.32)=0.261.

The first split is x<3.5.