Lecture 05 – Regression (KS33003 Machine Learning) Part 1

Vocabulary flashcards summarising essential terms and definitions from Lecture 05 on Regression, covering concepts such as linear models, supervised learning, least squares, R², and multivariate regression.

Regression

A predictive modeling technique that estimates the relationship between a dependent variable and one or more independent variables.

Linear Model

A family of statistical models that assume a linear relationship between inputs and output; includes linear and logistic regression.

Linear Regression

A linear model used to predict continuous numeric outcomes by fitting a line (or hyperplane) to the data.

Logistic Regression

A linear model variant used to predict categorical (often binary) outcomes via the logistic function.

Bias (Intercept, α)

The constant term in a linear equation representing the predicted value when all features are zero.

Slope / Coefficient (β)

The weight assigned to a feature in linear regression indicating the feature’s influence on the prediction.

Independent Variable (X)

Input feature(s) used to predict the dependent variable in regression.

Dependent Variable (Y)

The target or response value that a regression model seeks to predict.

Supervised Learning

A machine-learning paradigm where models learn from labeled input–output pairs to make future predictions.

Regression Problem

A supervised-learning task that predicts continuous values (e.g., prices, temperatures).

Classification Problem

A supervised-learning task that assigns discrete class labels to inputs (e.g., spam vs. not spam).

Simple Linear Regression

Regression with a single predictor and one response, producing a straight-line relationship.

Multiple Linear Regression

Regression with two or more predictors, resulting in a multivariate linear equation.

Multivariate Regression

A regression task involving multiple input variables; synonymous with multiple linear regression in this context.

Least Squares Regression Line (LSRL)

The line that minimizes the sum of squared vertical distances between observed points and the fitted line.

Method of Least Squares

An optimization approach that estimates coefficients by minimizing the sum of squared errors.

Interpolation

Using a regression line to estimate unknown Y values for X values within the observed range.

Hyperplane

A generalization of a line or plane to higher dimensions representing the fitted surface in multiple regression.

Weighted Sum

The linear combination of feature values multiplied by their respective coefficients plus bias.

Cost Function

A quantitative measure (e.g., mean squared error) of how poorly a model’s predictions match the true values.

Gradient Descent Algorithm

An iterative optimization method that adjusts coefficients to minimize the cost function.

R-Squared (Coefficient of Determination)

A statistic from 0 to 1 indicating the proportion of variance in Y explained by the model.

Goodness of Fit

An assessment of how closely a model’s predictions align with the observed data; often measured by R².

Correlation

A statistic ranging from −1 to 1 that quantifies the strength and direction of a linear relationship between two variables.

Training Dataset

Labeled data used to teach a supervised-learning model how to map inputs to outputs.

Prediction

The model’s estimated output for a given input once training is complete.

Feature Scaling / Standardisation

The process of rescaling predictors (e.g., to zero mean, unit variance) to improve learning performance.

Scikit-learn

A popular Python library providing machine-learning algorithms, including linear_model for regression.

Boston Housing Dataset

A benchmark dataset of 596 cases and 14 attributes used to predict median house prices in Boston.

Beta Sign Interpretation

Positive β implies the response increases with the feature; negative β implies an inverse relationship; β≈0 indicates little effect.