IS 170 Decision Tree & Random Forest

Decision Tree

A flowchart-like model that splits data into smaller groups repeatedly until it can’t split anymore

Root Node

first split (most important feature)

Decision Nodes

intermediate splits

Branches

outcomes of decisions

Leaf Nodes

final prediction

Gini Impurity

Measures how “mixed” a node is

Gini Value: 0

pure (all same class)

Gini Value (high: ~0.5)

mixed

Lower Gini

= better split

Root node

feature with lowest Gini

Key idea of decision tree:

Decision tree chooses the split with the lowest impurity

Concept of overfitting for decision tree:

• Gini decreases (looks better)

• causes overfitting to increase (very specific to dataset)

concept behind gini 0

• not always good

• performs poorly on new data

max_depth

Limits how deep tree grows

min_samples_leaf

Minimum data points in a leaf

Applications of Decision Tree

• Fraud detection

• Customer churn

• Risk analysis

• Employee retention

Random Forest

Combines multiple decision trees to improve performance

Decision Trees use: _____ ; Random Forest uses: _____

decision trees use all features ; random forest use: random subset of features