Data Science - Model Selection Part 2

Advantages of Decision Trees (3)

• Highly Interpretable

• Can work with both categorical and numeric data

• No scaling neccessary

Disadvantages of Decision Trees (2)

• Prone to overfit

• Sensitive to data variations

Gini Impurity Index (What is it and value meanings)

Shows how likely an observation will be misclassified

• 0 = Completely Pure = All elements are in the same class

• 5 = Impure = Elements are distributed among classes

Accuracy

How often is the model correct?

Precision

Of instances predicted as positive, how many are actually correct?

Recall

Out of all of the actual positives, how many did we get?

Pro and con of decision trees

Pro: Reduces overfitting

Cons: Gets slower with larger datasets

Bagging vs Boosting

Bagging: Combines predictions from all models

Boosting: Each model tries to fix errors of previous models

Con of K Means

Influenced by outliers

Ways to pick K

Elbow method: Look for a “bend” in the errors

Silhouette Score: Ranges from -1 to 1, with higher being better

Covariance in GMM

Shape of clusters

One way to estimate params in GMM

Maximum Likliehood Estimation (MLE)

• Estimates params to maximize probabilities