BE-530: Machine Learning in Python - Lecture 1 Introduction

Foundational vocabulary and concepts for BE-530 Lecture 1, covering ML definitions, biomedical data types, workflow standards, evaluation metrics, and framing strategies.

Machine Learning (ML)

The process of learning a mapping from features $X$ to outcomes $y$ that generalizes to unseen, future-like data.

Generalization

The performance of a model on new data that was not seen during training.

Supervised Learning Loop

The iterative process of choosing a model family, defining a loss function, optimizing parameters to minimize loss, and evaluating on held-out data.

Loss

A penalty for errors that is optimized during training to guide parameter updates.

Metric

A value used for decisions and reporting that summarizes model behavior for a specific goal.

Classification

A task type where the output is a discrete label, such as a "yes/no" disease diagnosis.

Regression

A task type where the output is a continuous number, such as a risk score or lab value.

Leakage (or Peeking)

A failure in ML workflow where information unavailable at prediction time is included in training, creating fake improvements.

Baseline

The simplest comparator (e.g., majority class for classification or mean/median for regression) used to see if ML adds value.

Unit of Analysis

The level at which data is defined and split, such as patient-level, visit-level, time-window level, or image-level.

Prevalence

The overall rate of the positive class in a dataset.

Prevalence Baseline

A baseline strategy that ignores all features and always predicts the most common class.

Train Set

The portion of data used to fit model parameters.

Validation Set

The portion of data used to tune hyperparameters and thresholds.

Test Set

A final, unbiased estimate used only once at the end of modeling; it must not be used to guide choices.

Confusion Matrix

A table showing the breakdown of error types: True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN).

Accuracy Trap

A situation under data imbalance where high accuracy is misleading because the model fails to detect the minority class.

Sensitivity (Recall or TPR)

A metric that measures how many actual positives the model catches, calculated as $\frac{TP}{TP + FN}$.

Specificity (TNR)

A metric that measures how many actual negatives the model correctly identifies, calculated as $\frac{TN}{TN + FP}$.

Precision (PPV)

A metric that measures how much to trust a predicted positive outcome, calculated as $\frac{TP}{TP + FP}$.

NPV (Negative Predictive Value)

A metric calculated as $\frac{TN}{TN + FN}$.

AUROC

The Area Under the Receiver Operating Characteristic curve, which plots the True Positive Rate (TPR) vs. the False Positive Rate (FPR) across thresholds.

Memorization

A state where a model has high training performance but low performance on new data.

Pipelines

A code structure (e.g., from Scikit-Learn) that reduces accidental leakage by ensuring preprocessing transforms are fit only on the training data.

End-to-end Machine Learning

Building a complete machine learning system that can go from data input to prediction output.

Defensible Workflow Components

Key elements of a machine learning process including framing, evaluation, reporting, and reproducibility.

Generalization

The ability of a model to perform well on unseen data, as opposed to just learning the training data.

Supervised Learning Loop

A process in machine learning where model parameters are adjusted based on training errors.

Loss Function

A measure used to evaluate how well a model's predictions match the actual outcomes.

TP (True Positive)

The number of positive cases correctly predicted by the model.

FP (False Positive)

The number of negative cases incorrectly predicted as positive by the model.

TN (True Negative)

The number of negative cases correctly predicted by the model.

FN (False Negative)

The number of positive cases incorrectly predicted as negative by the model.

Confusion Matrix

A table used to describe the performance of a classification model by highlighting true vs. predicted classifications.

Sensitivity (Recall)

The ability of a model to correctly identify positive cases.

Precision

The ratio of correctly predicted positive observations to the total predicted positives.

ROC Curve

A graphical representation of a model's diagnostic ability across all classification thresholds.

Cross-Validation

A technique used to assess how the results of a statistical analysis will generalize to an independent dataset.

Baseline Model

The simplest model used to compare the performance of more complex machine learning models.

Label Noise

The inaccuracies present in the labels of the dataset which can degrade model performance.

Data Leakage

The unintentional use of information from outside the training dataset during model training.

Feature Selection

The process of identifying and selecting a subset of relevant features for model training.

Hyperparameters

Settings or configurations that are set before training a model, influencing its learning process.

Reproducibility

The ability to obtain consistent results using the same methods and data across different experiments.

Biomedical Data

Data that is used in machine learning models pertaining to health and medical conditions.

Clinical Text Processing

The use of natural language processing techniques to analyze clinical texts.