Machine Learning - Linear Regression

These flashcards cover key vocabulary and concepts related to linear regression in machine learning.

Simple linear regression

A method to model the relationship between one independent variable (x) and a dependent variable (y) by fitting a linear equation.

Loss function

A function that measures the difference between predicted values (ŷ) and actual values (y).

L1 loss

Absolute-value loss defined as L1(y, ŷ) = |y - ŷ|, indicating the magnitude of prediction errors.

L2 loss

Squared-error loss defined as L2(y, ŷ) = (y - ŷ)², which emphasizes larger errors more than smaller ones.

Least squares

A method used in regression analysis that minimizes the sum of the squares of the residuals to find the best-fitting line.

Gradient descent

An iterative optimization algorithm used to minimize the loss function by updating weights incrementally based on the gradient.

Learning rate (α)

A hyperparameter that determines the size of the steps taken towards the minimum of the loss function during optimization.

Stochastic gradient descent (SGD)

A variant of gradient descent where the weights are updated using a randomly selected subset of training examples.

Multivariable linear regression

A type of regression analysis where two or more predictor variables are used to predict the outcome of a response variable.

Regularization

A technique used to prevent overfitting by adding a penalty to large coefficients in the loss function.