W2 - 2.3 - Ensemble, Model Evaluation

Decision Tree Induction

A classification method that works by a top-down, divide and conquer approach, which may not guarantee optimal solutions but often results in efficient classifications.

Greedy Approach

A problem-solving method that makes the locally optimal choice at each stage with the hope of finding a global optimum.

Bayesian Classification

A classification method based on applying Bayes' theorem, which provides a probabilistic foundation for classifying instances.

Incremental Classification

A characteristic of certain classification models, such as Bayesian classifiers, allowing them to update their model with new data without needing to be rebuilt entirely.

Support Vector Machines (SVM)

A supervised learning model that classifies data by finding the optimal hyperplane which separates classes in high-dimensional space.

Neural Networks

Complex models designed to mimic the way human brains work, effective at capturing non-linear relationships but difficult to interpret.

Ensemble Methods

Techniques that combine multiple models to improve classification accuracy, rather than relying on a single model.

Bagging

A type of ensemble method that aggregates the predictions of multiple models, treating them equally and using majority voting.

Boosting

An ensemble technique that combines multiple weak classifiers into a single strong classifier by adjusting weights based on misclassifications.

Confusion Matrix

A table used to evaluate the performance of a classification model, showing true positives, false positives, true negatives, and false negatives.

Sensitivity

The proportion of actual positives that are correctly identified by the classifier.

Specificity

The proportion of actual negatives that are correctly identified by the classifier.

Precision

The ratio of true positives to the sum of true and false positives, indicating the accuracy of positive predictions.

Accuracy

The overall proportion of correct predictions made by a classification model.

Cross-Validation

A technique for assessing how the results of a statistical analysis will generalize to an independent dataset, often using k-fold partitions.

Bootstrapping

A resampling technique that involves taking repeated samples from a dataset with replacement to assess variability.

False Positive

A mistake made by a classifier where a correct instance is labeled as positive when it is actually negative.

False Negative

A mistake made by a classifier where a positive instance is incorrectly labeled as negative.

Multi-Class Classification

Classification tasks where there are more than two classes to predict.

Exact Match

A strict evaluation criterion where a prediction is only considered correct if it exactly matches the true class label.