Machine Learning: Decision Trees for Classification

These flashcards cover key vocabulary and concepts related to decision trees in machine learning, particularly their structure, function, and methods for ensuring accuracy and preventing overfitting.

Decision Tree

A hierarchical structure made up of nodes and edges used for classification tasks.

Node

An object in a decision tree representing a question or decision point.

Edge

A directed link from a parent node to a child node.

Leaf

A node that has no outgoing edges to child nodes and signifies a final decision.

Root Node

The top node of a decision tree with no parent.

Depth of a Node

The number of edges that must be followed to reach that node from the root node.

Decision Tree Classifier (DTC)

A decision tree specifically built for classification tasks, where each leaf corresponds to a class.

Hyperplane

A decision boundary used to divide feature space in classification tasks.

Gini Impurity Measure

A measure of impurity in a node given by the formula: $Gini = P_0 (1-P_0) + P_1 (1-P_1)$.

Overfitting

A condition where a model learns the details of the training data too well, leading to poor generalization on unseen data.

Early Stopping

A technique used to prevent further growth of the tree based on node statistics, aimed at reducing overfitting.

Cost Complexity Pruning

A method to prevent overfitting by reducing the size of an already trained decision tree.

Probability Estimate in DTC

The proportion of instances from the predicted class in the resulting leaf, provided by the method $DTC.predict ext{-}proba(X)$.