Fundamentals of Machine Learning (4IDSS1101) Exam Prep

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/50

flashcard set

Earn XP

Description and Tags

Comprehensive vocabulary flashcards covering the key terms, statistical foundations, and mathematical models across all five modules of the Fundamentals of Machine Learning exam prep pack.

51 Terms

1
New cards

Machine Learning

A branch of AI in which computer systems learn patterns from data and improve at a task with experience, making predictions or decisions without being explicitly programmed with rules for every case.

2
New cards

Statistical Learning

A framework that focuses on inference: understanding relationships between variables using interpretable models, formal assumptions, and uncertainty quantification (confidence intervals, p-values).

3
New cards

Supervised Learning

A learning paradigm that learns from labelled examples (inputs xx with known outputs yy) to predict yy for new inputs xx, encompassing regression for continuous values and classification for discrete categories.

4
New cards

Unsupervised Learning

A learning paradigm that operates strictly on inputs without labels to uncover hidden structures, groupings, or dominant directions of variation (such as clustering or PCA).

5
New cards

Reinforcement Learning

A paradigm where an agent takes actions in an environment, receives feedback via rewards or penalties, and learns a policy that maximizes cumulative reward over time through trial and error.

6
New cards

Learning Model

A mathematical representation of the relationship between inputs (features) and outputs, with parameters learned from data, written as y≈f(x;θ)y \thickapprox f(x; \theta) and used to generate predictions.

7
New cards

Geometric Learning Model

A model that treats data points as vectors in feature space and utilizes geometry (distances, hyperplanes, margins, and decision boundaries) to separate or group them.

8
New cards

Probabilistic Learning Model

A model that represents uncertainty using probability distributions and outputs conditional class probabilities, typically applying Bayes' theorem.

9
New cards

Generalized Linear Model (GLM)

An extension of linear regression for response variables that are not normally distributed, structured into a random component, a systematic component, and a link function.

10
New cards

Random Component (GLM)

The component of a GLM specifying that the response variable YY follows a distribution from the exponential family (such as normal, binomial, or Poisson), determining how variance relates to the mean.

11
New cards

Systematic Component (GLM)

The linear predictor η=β0+β1x1+⋯+βpxp\eta = \beta_0 + \beta_1 x_1 + \dots + \beta_p x_p that linearly combines the model predictors in a Generalized Linear Model.

12
New cards

Link Function

A monotonic function g(μ)=ηg(\mu) = \eta that connects the expected response mean μ=E(Y)\mu = E(Y) to the linear predictor, ensuring predictions remain within a valid mathematical domain.

13
New cards

Raw Residual

The simple difference between the observed response and the fitted value in a GLM, expressed as ri=yi−μ^ir_i = y_i - \hat{\mu}_i.

14
New cards

Deviance Residual

A standardized measure of an individual observation's lack of fit in a GLM, defined as di=sign(yi−μ^i)×deviance contribution of observation id_i = \text{sign}(y_i - \hat{\mu}_i) \times \sqrt{\text{deviance contribution of observation } i}, such that ∑di2\sum d_i^2 equals total model deviance.

15
New cards

Over-dispersion

A condition in count models (such as Poisson regression) where the empirical variance of the response variable exceeds its mean, violating the theoretical assumption of equidispersion.

16
New cards

Parametric ML Method

A learning method that assumes a model of fixed functional form with a finite set of parameters that does not grow with the sample size, learning strictly by estimating those parameters.

17
New cards

Least-Squares Principle

A parameter estimation criterion that selects the intercept and slope to minimize the sum of squared residuals: ∑(yi−β0−β1xi)2\sum (y_i - \beta_0 - \beta_1 x_i)^2.

18
New cards

Regression Coefficient (βj\beta_j)

A model parameter representing the expected change in the response variable yy for a one-unit increase in predictor xjx_j, holding all other predictors constant.

19
New cards

Homoscedasticity

The classical linear regression assumption requiring the error terms to have a constant variance across all levels of the predictors, verified when residual plots show no funnel shape.

20
New cards

Multicollinearity

A statistical phenomenon in multiple regression where two or more predictor variables are strongly correlated, causing inflated standard errors and unstable coefficient estimates.

21
New cards

Variance Inflation Factor (VIF)

A diagnostic metric used to quantify the severity of multicollinearity in a multiple regression model, where values exceeding 55 to 1010 signal problematic collinearity.

22
New cards

Gauss–Markov Theorem

A mathematical theorem establishing that under standard regression assumptions, the ordinary least-squares estimators achieve the smallest variance among all linear unbiased estimators (BLUE).

23
New cards

Odds

The probability that an event occurs divided by the probability that it does not occur, defined as p1−p\frac{p}{1 - p}.

24
New cards

Odds Ratio

The ratio of the odds of an event occurring under one condition compared to another; in logistic regression, eβe^\beta is the factor by which the odds are multiplied per unit increase in the predictor.

25
New cards

Logit Transformation

The natural logarithm of the odds, defined as logit(p)=ln⁡(p1−p)\text{logit}(p) = \ln\left(\frac{p}{1 - p}\right), which maps probabilities from the interval (0,1)(0, 1) onto the entire real number line.

26
New cards

Maximum Likelihood Estimation (MLE)

An estimation technique that identifies parameter values that maximize the likelihood function, making the observed data most probable under the model.

27
New cards

Likelihood-Ratio Test

A hypothesis test comparing two nested models via LR=−2(ln⁡(L0)−ln⁡(L1))\text{LR} = -2(\ln(L_0) - \ln(L_1)), asymptotically following a χ2\chi^2 distribution with degrees of freedom equal to the number of omitted parameters.

28
New cards

McFadden's Pseudo R2R^2

A goodness-of-fit measure for logistic regression defined as 1−ln⁡(Lmodel)ln⁡(Lnull)1 - \frac{\ln(L_{\text{model}})}{\ln(L_{\text{null}})}, reflecting relative improvement in log-likelihood over a null model.

29
New cards

Hosmer–Lemeshow Test

A calibration test for logistic regression that sorts observations by predicted probabilities, partitions them into approximately 10 groups, and compares observed to expected event counts using a χ2\chi^2 test.

30
New cards

Separation (Logistic Regression)

A numerical issue in logistic regression where a predictor perfectly divides outcome classes, causing maximum likelihood estimates to diverge to ±∞\pm\infty and standard errors to explode.

31
New cards

Classification Tree

A tree-structured predictive model that predicts categorical class labels by recursively splitting data on feature values through root, internal decision, and leaf nodes.

32
New cards

Tree Depth

The total number of split levels in a decision tree, measured as the length of the longest path from the root node to a terminal leaf node.

33
New cards

Confusion Matrix

A 2×22 \times 2 contingency table comparing actual outcomes against predicted classes, tabulating true positives (TPTP), false positives (FPFP), false negatives (FNFN), and true negatives (TNTN).

34
New cards

Sensitivity

The proportion of actual positive cases correctly identified by a classifier, calculated as TPTP+FN\frac{TP}{TP + FN}; also termed hit rate or true-positive rate.

35
New cards

Specificity

The proportion of actual negative cases correctly identified by a classifier, calculated as TNTN+FP\frac{TN}{TN + FP}; also termed true-negative rate.

36
New cards

ROC Curve

A diagnostic plot showing sensitivity versus 1−specificity1 - \text{specificity} across all possible classification probability cut-offs, evaluated by the Area Under the Curve (AUC).

37
New cards

Lift Curve

A visual performance curve plotting the ratio of a model's hit rate to the hit rate of random selection across various targeting depths, where lift values greater than 11 outperform random choice.

38
New cards

McNemar Test

A paired statistical test evaluating two classifiers evaluated on the same test cases, using discordant errors bb and cc in χ2=(b−c)2b+c\chi^2 = \frac{(b - c)^2}{b + c} with 11 degree of freedom.

39
New cards

Bagging

An ensemble technique that draws multiple bootstrap samples from training data, fits an independent base model on each sample, and aggregates outputs via voting or averaging to reduce variance.

40
New cards

Random Forest

An ensemble method combining bagging with random feature sub-selection at each split, decorrelating the individual trees to achieve a more robust and lower-variance average.

41
New cards

Boosting

A sequential ensemble algorithm that iteratively trains base learners to correct the residual errors or misclassifications of earlier models, combining predictions into a weighted final result to reduce bias and variance.

42
New cards

Support Vector Machine (SVM)

A supervised learning method that classifies data by identifying the hyperplane wTx+b=0w^T x + b = 0 that maximizes the margin separating distinct classes.

43
New cards

Support Vectors

The critical training data points that lie directly on or within the margin boundaries (having dual multipliers αi>0\alpha_i > 0) and solely determine the position of the separating hyperplane.

44
New cards

Soft Margin

An SVM formulation that tolerates margin violations and misclassifications by incorporating non-negative slack variables ξi≥0\xi_i \ge 0 and a penalty parameter CC.

45
New cards

Slack Variables (ξi\xi_i)

Parameters in soft-margin SVM optimization that record the degree of violation when training instances fall on the wrong side of the margin or are misclassified.

46
New cards

Kernel Trick

A computational approach that replaces inner products xiTxjx_i^T x_j in dual formulations with a kernel function K(xi,xj)=ϕ(xi)Tϕ(xj)K(x_i, x_j) = \phi(x_i)^T \phi(x_j), enabling nonlinear decision boundaries without explicitly computing high-dimensional feature representations ϕ\phi.

47
New cards

Radial Basis Function (RBF) Kernel

A distance-based nonlinear kernel defined as K(x,z)=exp⁡(−γ∥x−z∥2)K(x, z) = \exp(-\gamma \|x - z\|^2), producing flexible local decision boundaries controlled by the parameter γ\gamma.

48
New cards

Support Vector Regression (SVR)

A regression technique adapted from SVMs that fits a function within an ε\varepsilon-insensitive tube, penalizing prediction errors only when they exceed a distance of ε\varepsilon.

49
New cards

ε\varepsilon-Insensitive Loss

A loss function defined as Lε(y,f)=max⁡(0,∣y−f(x)∣−ε)L_\varepsilon(y, f) = \max(0, |y - f(x)| - \varepsilon), which ignores errors smaller than ε\varepsilon and penalizes larger deviations linearly.

50
New cards

Principal Component Analysis (PCA)

An unsupervised linear dimension-reduction technique that computes orthogonal eigenvectors of the covariance or correlation matrix to project data onto principal components that maximize captured variance.

51
New cards

Clustering

An unsupervised learning task that groups unlabelled observations into clusters such that within-cluster points share high similarity while between-cluster differences are maximized.