07 - Regression

Classification: predicting a value belonging to a class

Regression: Predicting a value belonging to a continuous set

Linear Dependence: constant rate of increase of one variable with respect to another

Regression Analysis: describes the relationship between two or more variables

B0 in linear regression: estimated average value of Y when X is 0

B1 in linear regression: estimated change in the avg value of Y if we make a unit change in X

SST: overall variability in Y

SSR: Variation in Y explained by the regression line

SSE: variation in Y left unexplained

Hypothesis Testing: statistical measure of how certain we can be that there is a linear relatinoship in the data

R-squared is the fraction of the variation which is explained by the lienar regression model

Linear Regression does not work well for: linear models of parameters, outside the variable range, doesn’t map predictions to categorical variables

Logistic Regression: estimates the probability P(Y=1) for variable values X, example of an activation function

B0 in logistic model shifts the curve right or left, estimated average value of Y when X=0

B1 in logistic model controls how steep the S-Shaped curve is, estimates the change in the average value of Y if we make a unit change in X

Representation: choosing the functions that can be learned, the set of hypotheses

Evaluation: loss function for penalising errors

Gradient Descent makes small steps along a function to find a local minimum, by taking a step in the opposite direction to the gradient

Learning Rate A: deterimines the size of the adjustment made to each weight at each step in the process, too large an alpha, may overshoot the minimum, too small an alpha, might not converge on the local minimum

Batch GD: compute the total loss function as the average loss for each training example (sum over all training examples before we make updates) - very slow

Stochastic GD: choose a single training example, compute the loss and do the update directly (takes random steps but moves downhilll on average)