Support Vector Machines (Part 1) and Classification Evaluation

Vocabulary flashcards covering the core concepts of Support Vector Machines (SVM), hyperplanes, margins, the C parameter, feature scaling pipelines, and classification evaluation metrics.

Regression

A supervised learning task that predicts a continuous numeric target value (such as temperature or power demand) based on input features.

Classification

A supervised learning task that predicts a categorical class label (such as normal versus faulty) for an observation based on input features.

Hyperplane

A flat decision boundary in feature space that separates different classes, appearing as a line in two dimensions and as a higher-dimensional flat subspace when more features are present.

Support Vectors

The critical training observations that lie closest to the decision boundary (touching or entering the margin) and solely determine the position and orientation of the fitted boundary.

Maximum Margin

The criterion by which SVM chooses a decision boundary that remains as far as possible from the nearest training observations of both classes to ensure geometric robustness.

Hard Margin

An SVM optimization formulation that strictly forbids any misclassifications or margin violations, requiring all training points to satisfy $y_i(\mathbf{w}^T \mathbf{x}_i + b) \ge 1$.

Soft Margin

An SVM formulation that incorporates non-negative slack variables ($\xi_i$) into the objective function to permit and control margin violations when data contains noise or overlap.

Slack Variable ($\xi_i$)

A penalty variable in soft-margin SVM optimization that quantifies the degree by which a specific training observation violates the margin or crosses the decision boundary.

Regularization Parameter $C$

A hyperparameter in soft-margin SVM that balances margin width against violation penalties: a small $C$ tolerates more violations to produce a wider margin, while a large $C$ penalizes violations heavily to fit the training data closely.

StandardScaler

A preprocessing technique that standardizes continuous features to zero mean and unit variance, preventing large-magnitude measurement units from dominating distance-based SVM optimization.

Data Leakage

An evaluation error that occurs when information from validation or test data inadvertently influences training, avoided in SVM by fitting preprocessing scalers strictly within cross-validation folds.

LinearSVC

A scikit-learn linear SVM implementation based on liblinear that scales efficiently for large dense or sparse datasets but does not explicitly store support vectors.

One-Class SVM

An unsupervised or semi-supervised SVM variant used in fault detection to learn boundaries around normal machine operational data and flag anomalous deviations.

Confusion Matrix

A layout table used to evaluate classification performance where rows represent actual classes and columns represent predicted classes, partitioning results into TP, FP, FN, and TN.

Precision

The fraction of positive predictions that are truly correct, calculated across the predicted positive column as $\frac{\text{TP}}{\text{TP} + \text{FP}}$.

Recall

The fraction of actual positive instances successfully detected by the model, calculated across the actual positive row as $\frac{\text{TP}}{\text{TP} + \text{FN}}$.

$F_1$ Score

The harmonic mean of precision and recall calculated as $2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$, providing a single balance metric that ignores true negatives.

ROC Curve

A graphical plot representing classification ranking performance by plotting the true-positive rate (recall) against the false-positive rate across varying decision thresholds.

AUC (Area Under the Curve)

A scalar summary metric indicating the area under an ROC curve across all thresholds, where $1.0$ indicates perfect class ranking and $0.5$ represents random chance.

Classification Threshold

The cutoff value applied to continuous model prediction scores to assign binary class labels; lowering the threshold increases recall while generally decreasing precision.