1/50
Comprehensive vocabulary flashcards covering the key terms, statistical foundations, and mathematical models across all five modules of the Fundamentals of Machine Learning exam prep pack.
Machine Learning
A branch of AI in which computer systems learn patterns from data and improve at a task with experience, making predictions or decisions without being explicitly programmed with rules for every case.
Statistical Learning
A framework that focuses on inference: understanding relationships between variables using interpretable models, formal assumptions, and uncertainty quantification (confidence intervals, p-values).
Supervised Learning
A learning paradigm that learns from labelled examples (inputs x with known outputs y) to predict y for new inputs x, encompassing regression for continuous values and classification for discrete categories.
Unsupervised Learning
A learning paradigm that operates strictly on inputs without labels to uncover hidden structures, groupings, or dominant directions of variation (such as clustering or PCA).
Reinforcement Learning
A paradigm where an agent takes actions in an environment, receives feedback via rewards or penalties, and learns a policy that maximizes cumulative reward over time through trial and error.
Learning Model
A mathematical representation of the relationship between inputs (features) and outputs, with parameters learned from data, written as y≈f(x;θ) and used to generate predictions.
Geometric Learning Model
A model that treats data points as vectors in feature space and utilizes geometry (distances, hyperplanes, margins, and decision boundaries) to separate or group them.
Probabilistic Learning Model
A model that represents uncertainty using probability distributions and outputs conditional class probabilities, typically applying Bayes' theorem.
Generalized Linear Model (GLM)
An extension of linear regression for response variables that are not normally distributed, structured into a random component, a systematic component, and a link function.
Random Component (GLM)
The component of a GLM specifying that the response variable Y follows a distribution from the exponential family (such as normal, binomial, or Poisson), determining how variance relates to the mean.
Systematic Component (GLM)
The linear predictor η=β0+β1x1+⋯+βpxp that linearly combines the model predictors in a Generalized Linear Model.
Link Function
A monotonic function g(μ)=η that connects the expected response mean μ=E(Y) to the linear predictor, ensuring predictions remain within a valid mathematical domain.
Raw Residual
The simple difference between the observed response and the fitted value in a GLM, expressed as ri=yi−μ^i.
Deviance Residual
A standardized measure of an individual observation's lack of fit in a GLM, defined as di=sign(yi−μ^i)×deviance contribution of observation i, such that ∑di2 equals total model deviance.
Over-dispersion
A condition in count models (such as Poisson regression) where the empirical variance of the response variable exceeds its mean, violating the theoretical assumption of equidispersion.
Parametric ML Method
A learning method that assumes a model of fixed functional form with a finite set of parameters that does not grow with the sample size, learning strictly by estimating those parameters.
Least-Squares Principle
A parameter estimation criterion that selects the intercept and slope to minimize the sum of squared residuals: ∑(yi−β0−β1xi)2.
Regression Coefficient (βj)
A model parameter representing the expected change in the response variable y for a one-unit increase in predictor xj, holding all other predictors constant.
Homoscedasticity
The classical linear regression assumption requiring the error terms to have a constant variance across all levels of the predictors, verified when residual plots show no funnel shape.
Multicollinearity
A statistical phenomenon in multiple regression where two or more predictor variables are strongly correlated, causing inflated standard errors and unstable coefficient estimates.
Variance Inflation Factor (VIF)
A diagnostic metric used to quantify the severity of multicollinearity in a multiple regression model, where values exceeding 5 to 10 signal problematic collinearity.
Gauss–Markov Theorem
A mathematical theorem establishing that under standard regression assumptions, the ordinary least-squares estimators achieve the smallest variance among all linear unbiased estimators (BLUE).
Odds
The probability that an event occurs divided by the probability that it does not occur, defined as 1−pp.
Odds Ratio
The ratio of the odds of an event occurring under one condition compared to another; in logistic regression, eβ is the factor by which the odds are multiplied per unit increase in the predictor.
Logit Transformation
The natural logarithm of the odds, defined as logit(p)=ln(1−pp), which maps probabilities from the interval (0,1) onto the entire real number line.
Maximum Likelihood Estimation (MLE)
An estimation technique that identifies parameter values that maximize the likelihood function, making the observed data most probable under the model.
Likelihood-Ratio Test
A hypothesis test comparing two nested models via LR=−2(ln(L0)−ln(L1)), asymptotically following a χ2 distribution with degrees of freedom equal to the number of omitted parameters.
McFadden's Pseudo R2
A goodness-of-fit measure for logistic regression defined as 1−ln(Lnull)ln(Lmodel), reflecting relative improvement in log-likelihood over a null model.
Hosmer–Lemeshow Test
A calibration test for logistic regression that sorts observations by predicted probabilities, partitions them into approximately 10 groups, and compares observed to expected event counts using a χ2 test.
Separation (Logistic Regression)
A numerical issue in logistic regression where a predictor perfectly divides outcome classes, causing maximum likelihood estimates to diverge to ±∞ and standard errors to explode.
Classification Tree
A tree-structured predictive model that predicts categorical class labels by recursively splitting data on feature values through root, internal decision, and leaf nodes.
Tree Depth
The total number of split levels in a decision tree, measured as the length of the longest path from the root node to a terminal leaf node.
Confusion Matrix
A 2×2 contingency table comparing actual outcomes against predicted classes, tabulating true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN).
Sensitivity
The proportion of actual positive cases correctly identified by a classifier, calculated as TP+FNTP; also termed hit rate or true-positive rate.
Specificity
The proportion of actual negative cases correctly identified by a classifier, calculated as TN+FPTN; also termed true-negative rate.
ROC Curve
A diagnostic plot showing sensitivity versus 1−specificity across all possible classification probability cut-offs, evaluated by the Area Under the Curve (AUC).
Lift Curve
A visual performance curve plotting the ratio of a model's hit rate to the hit rate of random selection across various targeting depths, where lift values greater than 1 outperform random choice.
McNemar Test
A paired statistical test evaluating two classifiers evaluated on the same test cases, using discordant errors b and c in χ2=b+c(b−c)2 with 1 degree of freedom.
Bagging
An ensemble technique that draws multiple bootstrap samples from training data, fits an independent base model on each sample, and aggregates outputs via voting or averaging to reduce variance.
Random Forest
An ensemble method combining bagging with random feature sub-selection at each split, decorrelating the individual trees to achieve a more robust and lower-variance average.
Boosting
A sequential ensemble algorithm that iteratively trains base learners to correct the residual errors or misclassifications of earlier models, combining predictions into a weighted final result to reduce bias and variance.
Support Vector Machine (SVM)
A supervised learning method that classifies data by identifying the hyperplane wTx+b=0 that maximizes the margin separating distinct classes.
Support Vectors
The critical training data points that lie directly on or within the margin boundaries (having dual multipliers αi>0) and solely determine the position of the separating hyperplane.
Soft Margin
An SVM formulation that tolerates margin violations and misclassifications by incorporating non-negative slack variables ξi≥0 and a penalty parameter C.
Slack Variables (ξi)
Parameters in soft-margin SVM optimization that record the degree of violation when training instances fall on the wrong side of the margin or are misclassified.
Kernel Trick
A computational approach that replaces inner products xiTxj in dual formulations with a kernel function K(xi,xj)=ϕ(xi)Tϕ(xj), enabling nonlinear decision boundaries without explicitly computing high-dimensional feature representations ϕ.
Radial Basis Function (RBF) Kernel
A distance-based nonlinear kernel defined as K(x,z)=exp(−γ∥x−z∥2), producing flexible local decision boundaries controlled by the parameter γ.
Support Vector Regression (SVR)
A regression technique adapted from SVMs that fits a function within an ε-insensitive tube, penalizing prediction errors only when they exceed a distance of ε.
ε-Insensitive Loss
A loss function defined as Lε(y,f)=max(0,∣y−f(x)∣−ε), which ignores errors smaller than ε and penalizes larger deviations linearly.
Principal Component Analysis (PCA)
An unsupervised linear dimension-reduction technique that computes orthogonal eigenvectors of the covariance or correlation matrix to project data onto principal components that maximize captured variance.
Clustering
An unsupervised learning task that groups unlabelled observations into clusters such that within-cluster points share high similarity while between-cluster differences are maximized.