CAP4611 - Decision Trees (L3.pdf)

supervised learning

use data to find a model that can accurately predict the labels of new examples

class label

classification

supervised learning on categorical labels

splitting rules

“if-else” decisions based on features

decision tree

a nested sequence of splitting rules with a class label as a return value

decision stump

simple decision tree with 1 splitting rule based on thresholding 1 feature

classification accuracy

most intuitive score; “If we use this rule, how many examples do we label correctly?”

mode

baseline rule predicts the ___ (no split)

training phase

use feature matrix ‘X’ and label vector ‘y’ to find a ‘model’

prediction phase

given an example x_i, use ‘model’ to predict a label ‘yhat_i’ (e.g. “sick” or “not sick’)

training error

fraction of times our prediction yhat_i does not equal the true y_i label

greedy recursive splitting

use a sequence of stumps to fit a tree

information gain

splitting score based on decreasing entropy