Decision Tree

• Split Search - Examines all possible binary splits of the data along each predictor variable to select the split that most reduces some measure of node entropy.
• What does the circles represent in a decision tree - decision
• What does the rectangles represent in a decision tree - tests
• Circles - terminal (leaf) nodes
• Top or starting node - root node
• Internal nodes - rectangles
• Information Gain(IG)- Is based on the reduction in entropy of a target variable after a dataset is split on an attribute
• Information Gain Formula - IG(Y,X) = E(Y) - E(Y|X)
• What measure is information gain based on - entropy
• Entropy - expected amount of information
• How much information does ‘an attribute‘ give about ‘the class (a target variable)‘? - Attributes that perfectly partition should give maximal information
• What is Entropy the measure of? - Measure of impurity(disorder)
• What is Entropy denoted as? - H(x) or E(x)
• What is the outcome if entropy = 0 - the outcome is ‘certain‘
• What is the outcome if entropy = maximum - the outcome is equally possible
• Entropy Formula -

  - Σ P(x)log2P(x)

\

• Information value - information content of an event E exponentially increases as the probability of occurring an event E ‘p(E)‘ decreases
• Information Value Formula - log2(1/p(E))
• What does the base represent in information value - the unity of information(possibility of selection)
• What is the best way to figure out the best attribute - information gain
• What is used to calculate information gain - entropy
• Modeling essentials - determine type of prediction, select useful inputs and optimize complexity
• Modeling essentials for decision trees - prediction rules, split search and pruning
• Maximal Tree - The most complex model within a sequence of models with increasing complexity.
• Steps to Pruning Two Splits in Maximum Tree- rate each subtree using validation assessment, select the subtree with the best assessment rating
• Steps for Subsequent Pruning - Continue pruning until all subtrees are considered, compare validation assessment among tree complexities
• Validation Assessments - Decision Optimization(Accuracy), Estimate Optimization(Squared Error)
• With binary target which two decision types do you consider - primary decision, secondary decision
• Minimize squared error - squared difference between target and prediction [(target - estimate)^2]
• 1-p - the probability of the event not occuring
• p/1-p - the odds of the event happening
• Properties of odds - is not symmetric, varying from 0 to infinity, when it’s 1 the probability is 50%
• Properties of log odds - is symmetric, going from minus infinity to positive infinity(like a line), when its 0 the probability is 50%, highly negative for low probabilities and highly positive for high probabilities.

\