Machine Learning Concepts

Bias-variance tradeoff

Describes the two sources of error in a model that need to be balanced and accounted for. 
Bias: comes from a model being too simple to capture the true pattern in the data, leads to underfitting.
- Detected by high training error AND validation error
- Fix:
     - Use more complex model
     - Add more/better features
     - Reduce regularization
Variance: comes from a model being fitted too closely to training data such that it's not able to generalize to real world data, leads to overfitting.
- Detected by low training error but high validation error
- Fix:
     - Get more training data
     - Simplify model (fewer features, shallower tree, etc)
     - Add regularization
     - Feature selection / dimensionality reduction
     - Bagging > Boosting  

b

b

c

c

d

d