Multiple Linear Regression

Aim of multilinear regression

To predict the score of an interval variable from multiple interval variable predictors.

F test function

Tests whether any of the independent variables in the model are significant

SStotal

The total variation (spread) of the data, calculated by summing the squared differences from the overall mean

SSmodel

Measures how far the predicted value is from the overall mean, calculated by summing the squared differences of the predicted value from the overall mean → spread explained by the model

SSresidual

Squared deviations of actual scores from the predicted values on the regression line → unexplained spread

R² (coefficient of determination)

The proportion of variance in the dependent variable that is predictable from the independent variables, calculated as SS_M/SS_T

Effect sizes R²

0.01 = small, 0.09 = medium, 0.25 = large

Ratio of cases to predictors (sample size)

At least 10-15 cases per predictor

Assumptions of MLR

Independent observations, normality, no outliers, homoscedasticity, linearity between DV and predictor, no multicollinearity

Independent observations

Data points must be independent; one person's score should not influence another's.

What to do if observations are not independent?

• Multilevel models

• Repeated measures ANOVA if there are multiple measurements of DV per case

Normality

Error terms must be normally distributed within the population.

What to do if the assumption of normality is violated?

Transform the dependent variable - either log transformation or square root transformation

Homoscedasticity

The assumption of equal variance for all predicted scores. Residuals should be evenly distributed along the line e = 0 on the plot

Linearity

The relationship between predictors and outcome must be linear.

Multicollinearity

Phenomenon where one predictor variable can be linearly predicted from the others with a substantial degree of accuracy

Diagnosis of multicollinearity using tolerance

Tolerance <.1 implies serious problem
Tolerance <.2 implies potential problem

Diagnosis of multicollinearity using VIF

VIF>10 implies serious problem
VIF>5 implies potential problem

How to fix multicollinearity

1. Increase sample size

2. Combine predictors

3. Remove predictor

Outliers in y space

• |z| > 3.3 is an outlier in y space

• Check standardised residuals

Outlier in x space

• Check mahalanobis distance

• Outlier if MD = 10 + 2*(#predictors)

Outliers in xy space

• Check cook’s distance

• Outlier in xy space if cooks distance > 1