Machine Learning Techniques - DAC 310 Final

Definitions, advantages, disadvantages, etc.

Decision Tree (Supervised)

A model that splits data into regions based on rules.

Used for classification and regression

• Advantages: Interpretable, handles nonlinearity, handles missing values (no missing values in the dependent variable; delete the value), easy to explain to management, and handles interaction effects 

• Disadvantages: No way to represent linear functions, have the tendency to overtrain (brittle trees).

Random Forest (Supervised)

A collection of trees. An ensemble technique

Used for classification and regression

• Advantages: Accurate, robust 

o   Determine variable importance, handle missing values, and ensembles are highly accurate with predictions 

• Disadvantages: Less interpretable 

o   Only useful for prediction and variable importance, very large PMML file 

K Nearest Neighbor (Supervised)

Predicts based on the closest K observations.

Used for classification and regression

·       Advantages: Easy, no training time (lazy learner), effective on non linear data

·       Disadvantages: slow, computation heavy predictions, high memory usage, sensititive to noise, scaled or high dimensional data

K Means (Unsupervised)

Clustering algorithm minimizing distance to cluster centers

Advantages: Fast computation

Disadvantages: Rigid, cannot correct later for erroneous decisions made earlier, sensitive to noise and outliers

Neural Networks (supervised)

“Black Box Technique”. Not used for explanatory purposes. a machine learning model inspired by the human brain, using interconnected layers of artificial neurons to process data, identify complex patterns, and make decisions. They are the foundation of deep learning, powering AI applications like computer vision, speech recognition, and natural language processing.

Classification and regression

High accuracy, automatic feature extraction, adaptability

Disadvantages: high computational costs, lack of interpretability (‘black box’), high data dependency

Genetic Algorithms

Optimization inspired by evolution/biology. Type of search technique. uses concepts of “natural selection” and “genetic inheritance”.

Advantages: Useful for very large search spaces, inherently parallel; easily distributed

Disadvantages: Finding large primes (not continuous)