Multiple Linear Regression

Simple linear regression

Simple linear regression

Has only one x and one y variable

Whats the difference btw multiple and

Differnece of number of variables we are including

Multiple linear regression

Has one y and two or more x variables

Example of simple linear regression

We predict rent based on square feet alone

Example of multiple linear regression

Predict rent based on square feet and age of the building

if more than one independent variable is to be used in the regression model, linear regression can be extended to

Multiple regression to accommodate to several independent variables for prediction

What happens when we add more variables

• adding more predictive power to model

• R and R2 will either stay the same or improve

• Residuals are generally closer to 0

Adding more independent variables to a model tends to

Improve prediction

Residuals are closer to 0 because we minimuze

epsilon, random error.

Categorical variable

Type

Impact of carat weight on diamond price (regression results)

We expect price goes up by 5333.86 on average

Multiple regression analysis additional statement (IMPROTANT)

controlling for the effect of other variables ; keeping other variables constant ; ceteris paribus ( ALL THIS MEANS Incremental impact on that specific variable)

Estimation power of ANOVA test (on sldies)

Reject null hypothesis and conclude that at least one of the models variables is significant (significance is less than .05) (on the price of diamonds)

Is it approproiate to interpret? (analyzing each coefficient)

if p vall is greater than .5 and 0 is in the range

Adjust R square is a modified version of R-Squared that

has been adjusted for the number of predictors in the model

For multiple regression analysis we use

ADjusted R2

R2 indicates how well

actual data points fit a line but adjusts for the number of terms in a model

The adjusted R-Squared value increases only when the new term

improves the model fit more than would be expected by chance

The adjusted R-Squared value decreases when a

Predictor improves the model by less than expected

R squared value always increases when

The number of variables increases

R -Squared values will still increase if you add a useless

variable to the model

Adjusted R-Squared value will

never increase if you add a useless X-Variable to the model

Adjusted R-Squared is mainly used for

the model selectiom but not the R-Squared value is used

Multicollineraity is a statistical phenomenon that occurs when

Two or more predictor variables in a regression model are highly correlated

Consequences of Multicollinearity

makes it challenging to determine the individual effect of each variable on the dependent variable

Multicollinearity may appear that some variables are

not significant when they might be individually, and vice versa

If multicollinearity is present in a regression model, it does not necessarily invalidate the model but it can

affect the precision of coefficient estimates. May lead to unstable and unpredictable coefficient values