Multiple Regression

Multiple regression is

a method use to predict one variable (Y-outcome) from more than one variable (X-predictor)

Both simple and multiple regression are

univariate

univariate means

only have 1 Y outcome

multivariate means

have more than 1 Y outcome

It is possible to have

a non-continuous X predictor (not in this class)

Both regression model (simple and multiple) can be capture using

path diagram

In path diagram, boxes are

observed variables (Xs & Y)

In path diagram, arrows are

causal hypotheses (X → Y means we believe X cause Y)

Side note: regression models simply model the causal process but

we actually can’t get cause-effect estimates just by using regression (only true experiment can get us there)

Simple regression path diagram has

1 intercept and 1 slope

Interpretation of intercept in simple regression

The value of Y, when X = 0

Interpretation of slope in simple regression

The change in Y given 1 point increase in X

Multiple regression path diagram has

1 intercept and #slope = #predictors

Interpretation of intercept in multiple regression

The value of Y when all predictors (X’s) is equal to 0

Interpretation of slope in multiple regression

The change of Y given 1 point increase in X1/X2…. holding all the other X’s constant

We don’t do 2 simple regressions when have 2 predictors bc

we want to see the unique relationship between each predictor and the outcome

If X1 and X2 have exactly r = 0 (no correlation) then that won’t be a problem doing 2 simple regression, but..

if X1 and X2 do correlates, then we aren’t going to see the unique relationship

EX: Age and Education level as predictors of the ACT

If you did them separately then you would think both were good predictors of ACT scores bc age and education level are obviously correlated (older → higher education)

BUT turns out,

only education truly predicts ACT scores (when put in multiple regression model)

only education truly predicts ACT scores because

the part of age that’s related to ACT scores was redundant with education level

The multiple R-squared in multiple regression explained

the variability of the outcome from both the predictors (ex: 24.9% of variation in ACT scores is explained by age and education together)

you can use as many predictors but it’s best

to keep it simple

When making a multiple regression model, the assumption is that all important variables are in the model, but

should also keep in mind that there might be potential alternative explanations when building our model

you can have multiple predictors in making our prediction, all the variables just have to be

significant when finding the p-value in multiple regression. (ex: if job experience and test scores both predict job performance, then we get both from applicants and use the formula and get better prediction than one alone)