Gradient Descent and Feature Selection

Gradient Descent

An iterative optimization algorithm used to minimize a loss function by adjusting model parameters.

Loss Function

A function that measures the difference between predicted values and actual values.

Learning Rate (alpha)

A hyperparameter that determines the size of the steps taken towards the minimum in gradient descent.

Local Minima

A point where the loss function is lower than neighboring points, but not the lowest possible.

Global Minima

The point at which the loss function achieves its lowest value across all possible points.

P-value

A statistical measure that indicates the probability that the observed data would occur by chance.

Statistical Significance

The likelihood that a relationship between variables is caused by something other than random chance.

Mean Squared Error (MSE)

A common loss function that measures the average of the squares of the errors.

Multiple Linear Regression

A regression model