1/12
Looks like no tags are added yet.
What is the Predictive Model Structure?
Root or Internal Node: Represents a feature
Leaf Node: Represents the target value
Branch: represents a decision rule
What is a Classification Tree?
A tree where the target variable takes categorical values (e.g. Male, Female)
What are Regression trees?
Trees that the target variable takes continuous values (e.g. Temperature)
How do you construct a decision tree?
Given a dataset, the algorithm:
→ Searches for the best rules to split samples that are dissimilar
→ Repeats the process until the partitions (leaves) reach a certain degree homogeneity (the state of being uniform)
What are the two main methods to split samples?
Gini Index (Impurity)
Information Gain
What is Gini Index (Impurity)?
Used in Classification and Regression Trees (CART)

What is Information Gain?

What is the formula for Entropy {H(Y)}?

What is the formula for Conditional Entropy {H(Y|A)}?

After calculating all Information gains, where do you split?
The feature that gives the largest Information Gain
What are limitations of Decision Trees?
Unstable: A small change in the data can lead to a large change in the structure of the obtained decision tree
Greedy Search: Tree construction has no backtracking, as searching in the space of ALL possible trees is unfeasible
Relatively Inaccurate: Many other models, such as Support Vector Machines and Neural Networks, often perform better on similar data
What is Mutual Information?
Information gain
What is the Mutual Information Feature Selection Algorithm?
