Machine Learning - Decision Tree Learning

These flashcards contain key vocabulary and definitions related to decision tree learning in machine learning.

Decision Tree

A tree-like model used for classification and regression, where each internal node represents a feature and each leaf node represents a class label.

Entropy

A measure of disorder or impurity in a set of examples, used to assess the quality of splits in a decision tree.

Information Gain

The expected reduction in entropy resulting from a split on an attribute, used to determine the best attribute for decision making.

Overfitting

A modeling error that occurs when a decision tree is too complex and fits the training data too well, failing to generalize to unseen data.

Pruning

The process of removing subtrees from a decision tree to reduce complexity and improve accuracy on test data.

ID3 Algorithm

An algorithm invented by J. Ross Quinlan in 1979 that builds decision trees using Information Gain to select the most useful attributes.

Leaf Node

A terminal node in a decision tree that provides a classification decision.

Disjunctive Normal Form (DNF)

A standard way of expressing a logical formula as a disjunction of conjunctions, which can be derived from decision trees.

Bias in Decision-Tree Induction

The tendency of decision tree algorithms to prefer simpler trees with less depth, influenced by the method of attribute selection.

Continuous Features

Real-valued features that can be split within specified ranges in a decision tree.