simple linear regression

linear regression

straight line that best predicts an outcome (DV) from a predictor (IV):

y_i = b₀ + b₁x_i + ε_i

intercept

predicted value for DV when IV = 0 (where line crosses the y-axis i.e. mean of y)

slope

change in DV for every 1-unit increase in IV (relationship between x and y)

standardised slope

a 1-SD increase in IV predicts a β-SD change in DV (calculated by standardising both variables; β = r)

makes predictors with different scales comparable

residual

observed minus predicted value for person i (i.e. how far each actual point sits from the line → what the model gets wrong)

means model

model with no predictors → predicts everyone’s score with the mean of y i.e. a flat line (b1 = 0)

regression model

mode with predictors → predict from the regression line i.e. line has a slope (b1 ≠ 0)

sum of squares

measure of variation that, for each person, takes a deviation, squares it - to avoid cancelation of opposite values - and adds them up to get the total amount of variation

SS total

total error of the means model (squared difference between observations and mean line → quantifies how much y varies in total)

SS residual

error left over with the regression model (squared difference between observations and regression line → quantifies how much each data point deviates from the regression line)

SS model

the error you got rid of by using variable x as predictor (i.e. the model improvement)

method of least squares

the best regression line is the one that makes the sum of squared residuals (SS_r) as small as possible

F-statistic

explained variance divided by unexplained variance, each per degree of freedom:

• F < 1: worse than the means model

• F > 1: better than the means model

A large F with a significant p-value means the model with the predictor describes the data significantly better than the model without the predictor

multiple R-square

the proportion of variance in y that is explained by x in the sample (range: 0 to 1; in simple regression R² = r²)

SS_m / SS_t

adjusted multiple R-square

estimates how much variance the model would explain in the population by correcting for sample size and number of parameters

t-test

each predictor gets a test to assess whether the slope significantly differs from zero

t = b / SE(b)

with one predictor same as F-test

assumptions

unique:

• linearity between IV and DV

• normality of residuals

• homoscendasticity (equal spread of residuals)

general:

• independent residuals

• no outliers