Tree-Based Methods

Flashcards summarizing key concepts related to tree-based methods in statistical learning, including decision trees, regression and classification techniques, and ensemble methods.

What are tree-based methods in statistical learning?

Methods that segment the predictor space into simple regions for regression and classification.

What is a decision tree?

A model used to represent the set of splitting rules used to segment the predictor space.

What is the main advantage of tree-based methods?

They are simple and useful for interpretation.

What are the two steps in the process of building a regression tree?

1. Divide the predictor space into distinct regions. 2. Make predictions based on the mean response in each region.

What is recursive binary splitting?

A top-down, greedy approach used for constructing regression trees by iteratively splitting the predictor space.

What does pruning a tree involve?

Reducing the complexity of a large tree to prevent overfitting, while maintaining predictive performance.

How are terminal nodes defined in a regression tree?

Regions in the predictor space where predictions are made based on the mean response of training observations.

What is the Gini index used for in classification trees?

A measure of total variance across classes, indicating node purity.

How does a classification tree differ from a regression tree?

A classification tree predicts the most common class in a region, while a regression tree predicts a continuous value.

What are ensemble methods?

Approaches that combine multiple models to improve prediction performance, such as bagging, random forests, and boosting.

What is the purpose of bagging?

To reduce the variance of a statistical learning method by averaging predictions from multiple models.

What might make decision trees less robust?

A small change in data can lead to large changes in the final estimated tree.

What is the role of cost complexity pruning in building trees?

It helps select a subtree that minimizes the test error rate by controlling tree complexity.

What is the main disadvantage of decision trees compared to other methods?

They typically do not achieve the same level of prediction accuracy as other supervised learning approaches.

What is the importance of node purity in classification trees?

Higher node purity means that the predicted outcomes are more certain and accurate for new observations.

Why might decision trees be preferred despite their limitations?

They are easier to interpret and can be visually represented.