PSY301 Lecture 5: Multiple Regressions

Multiple Regression

The prediction of scores on a single criterion (Y) based on scores on multiple predictors (X1,X2,….) simultaneously.

• More predictors included → better prediction of outcome (can account for more variance).

• But model becomes more complex.

• Predictors may not only be correlated with the criterion, but also be correlated with one another

• Sometimes, the predictors in a multiple regression do not overlap in their prediction of the criterion.

• Often, the predictors do overlap in their prediction of the criterion

In multiple regression, we can test for:

The overall relationship between Y and all predictors, collectively.

• R, R2, adjusted R2

• Tested for significance with an F test

Magnitude of relationship between Y and each individual predictor

• b, β, r (r 2 ), pr (pr2 ), sr (sr2 )

• Tested for significance with a t-test

We calculate a separate b, β, r, pr and sr for each individual predictor in the model.

Multiple Regression - Overall relationship

Multiple correlation coefficient (R)

• The overall relationship between Y and all predictors

Coefficient of multiple determination of simply “Rsquared” (R2 )

• The variance in Y jointly accounted for by all predictors

Adjusted R2

• Adjusted, more conservative estimate of R2

Multiple Regression - Individual predictor-criterion relationships

• b (unstandardised regression coefficient)

• β (standardised regression coefficient)

IN multiple regression, these coefficients are adjusted to account to any overlap between the predictors (Intercorrelations), especially in their shared prediction of Y.

Beyond the b and β coefficients, we can also calculate three additional estimates of the relationship between each predictor and Y.

Along with r, in multiple regression we also calculate two other correlation coefficients adjusted to estimate the unique relationship between any given predictor (Xj) and Y (by partialling out the variance explained by the other predictors).

• Partial correlation (pr)

• Semi-partial correlation (sr

Multiple Regression - Individual predictor-criterion relationships - Zero-order correlation (r)

• Normally don’t report

• The unadjusted correlation coefficient between any given predictor (Xj) and Y, ignoring the other predictors in the model.

• This would be the β if we were to run a bivariate regression with Xj and Y alone.

• We square this to get the variance in Y that can be explained by Xj, ignoring any other variables.

• The problem with relying on r (and r 2 ) is that it does not account for the intercorrelations between all the predictors in the model and their shared contributions in predicting Y.

Multiple Regression - Individual predictor-criterion relationships - Partial correlation (pr)

• The correlation between any given predictor (Xj) and Y after the variance accounted for by the other predictor(s) in the model has been partialled out of both Xj and Y.

• pr2 : Proportion of residual variance in Y that can be explained by Xj.

• Residual variance: left-over variance in Y that cannot be explained by other predictors.

Multiple Regression Individual predictor-criterion relationships • Semi-partial correlation (sr)

• The correlation between any given predictor (Xj) and Y after the variance accounted for by the other predictor(s) in the model has been partialled out of Xj.

• sr2 : Proportion of total variance in Y that can be explained by Xj.

Multiple Regression Individual predictor-criterion relationships - pr2 VS sr2

Multiple Regression The math behind it (correlated predictors)

The math behind it (uncorrelated predictors)

Bivariate vs Multiple Regression

Bivariate regression

• Linear model is a “line of best fit”

• Regression equation: 𝒀 = 𝒃𝑿 + 𝒂

Multiple regression

• Linear model is a “plane of best fit”

• Multiple regression equation (for two predictors): 𝒀 = 𝒃𝟏X1 + 𝒃𝟐𝑿2 + a

Reading a model in the right order

Is the model any good?

• Look at the F-test (is it significant?) and R2 (How much variance is explained?)

Which predictors matter?

• Look at each predictor’s significance and β. A significant β means it adds something unique.

How much does each add?

• Look at sr2 for each predictor, its unique share of the variance in the outcome.

When can we trust the model? - Assumption check

Linear relationships

• The predictors relate to the outcome in straight-line fashion, not curves.

Independent observations

• Each person’s data is separate, not duplicated or clustered cases.

Well-behaved residuals

• The leftovers errors are roughly normal and evenly spread across predicted scores.

Predictors not redundant

• No predictor is almost a copy of another (severe overlap muddies each one’s role)

Enough data

• More data is better, consider power analysis

Multiple regression: Keep in mind

Prediction is not causation

• The model shows which predictors track the outcome – not that changing them would change it.

The model only knows what it’s given

• A missing but important predictor can distort the roles of the ones included.

Bigger R2 isn’t always a better model

• A model can fit your sample beautifully and still fail on new data – watch adjusted R2