Multiple Regression

Why is a simple one-factor linear regression model often inadequate in finance and economics?

Financial and economic relationships are often influenced by multiple variables.

What does multiple regression allow an analyst to do?

Consider multiple independent variables at the same time.

What are three common uses of multiple regression models?

Identify relationships between variables, forecast variables, and test existing theories.

How can multiple regression identify relationships between variables?

By estimating how several factors are related to a dependent variable.

How can multiple regression be used for forecasting?

By using independent variables to predict a dependent variable.

How can multiple regression test theories?

By evaluating whether certain variables explain an outcome after controlling for other factors.

What is the general multiple linear regression model?

Yᵢ = b₀ + b₁X₁ᵢ + b₂X₂ᵢ + ... + bₖXₖᵢ + εᵢ.

In the general model, what does Yᵢ represent?

The ith observation of the dependent variable.

In the general model, what does Xⱼ represent?

An independent variable.

In the general model, what does Xⱼᵢ represent?

The ith observation of the jth independent variable.

In the general model, what does b₀ represent?

Intercept term.

In the general model, what does bⱼ represent?

Slope coefficient for an independent variable.

In the general model, what does εᵢ represent?

Error term for the ith observation.

In the general model, what does n represent?

Number of observations.

In the general model, what does k represent?

Number of independent variables.

What does the regression methodology estimate?

The intercept and slope coefficients.

What does multiple regression minimize?

The sum of squared error terms.

What is the formula for the sum of squared errors?

Σᵢ₌₁ⁿ εᵢ².

What is the estimated multiple regression equation?

Ŷᵢ = b̂₀ + b̂₁X₁ᵢ + b̂₂X₂ᵢ + ... + b̂ₖXₖᵢ.

What does the hat symbol indicate in regression output?

An estimate.

What does Ŷᵢ represent?

Predicted value of the dependent variable.

What does b̂₀ represent?

Estimated intercept.

What does b̂ⱼ represent?

Estimated slope coefficient.

What is a residual?

The difference between the observed value and the predicted value.

What is the formula for a residual?

ε̂ᵢ = Yᵢ − Ŷᵢ.

What is the expanded formula for a residual?

ε̂ᵢ = Yᵢ − (b̂₀ + b̂₁X₁ᵢ + b̂₂X₂ᵢ + ... + b̂ₖXₖᵢ).

If the residual is positive, what does that mean?

The actual value is above the predicted value.

If the residual is negative, what does that mean?

The actual value is below the predicted value.

If the residual is zero, what does that mean?

The prediction equals the observed value.

What is the null hypothesis commonly tested for a regression coefficient?

The slope coefficient equals zero.

What does a slope coefficient of zero imply?

No linear relationship with the dependent variable, holding other variables constant.

What is a p-value?

The smallest significance level at which the null hypothesis can be rejected.

How is coefficient significance tested using a p-value?

Compare the p-value to the chosen significance level.

What decision is made if p-value < significance level?

Reject the null hypothesis.

What decision is made if p-value > significance level?

Do not reject the null hypothesis.

What does rejecting the null hypothesis for a coefficient suggest?

The variable is statistically significant.

What does failing to reject the null hypothesis for a coefficient suggest?

The variable is not statistically significant.

How is the intercept interpreted in multiple regression?

Expected value of the dependent variable when all independent variables equal zero.

How is a slope coefficient interpreted in multiple regression?

Expected change in the dependent variable for a one-unit increase in that independent variable, holding the other independent variables constant.

Why are multiple regression slope coefficients called partial slope coefficients?

They measure the effect of one independent variable while holding the others constant.

What phrase is essential when interpreting a multiple regression slope coefficient?

Holding the other independent variables constant.

Why can a coefficient change when another independent variable is added?

The model now controls for the added variable.

If an independent variable's coefficient is positive, what does that imply?

The dependent variable is expected to increase as that variable increases, holding others constant.

If an independent variable's coefficient is negative, what does that imply?

The dependent variable is expected to decrease as that variable increases, holding others constant.

If a slope coefficient is 2.5, how should it be interpreted?

A one-unit increase in that variable is associated with a 2.5-unit increase in the dependent variable, holding other variables constant.

If a slope coefficient is −0.3, how should it be interpreted?

A one-unit increase in that variable is associated with a 0.3-unit decrease in the dependent variable, holding other variables constant.

What is the first assumption of multiple linear regression?

A linear relationship exists between the dependent variable and the independent variables.

What is the second assumption of multiple linear regression?

The residuals are normally distributed.

What is the third assumption of multiple linear regression?

The variance of the error terms is constant for all observations.

What is the fourth assumption of multiple linear regression?

The residual for one observation is not correlated with the residual for another observation.

What is the fifth assumption of multiple linear regression?

The independent variables are not random, and there is no exact linear relationship between two or more independent variables.

What does constant variance of error terms mean?

Homoskedasticity.

What is the opposite of homoskedasticity?

Heteroskedasticity.

What does uncorrelated residuals mean?

No serial correlation among residuals.

What does no exact linear relationship among independent variables mean?

No perfect multicollinearity.

Why are residual plots used?

To provide preliminary evidence of possible violations of regression assumptions.

What should a residuals-versus-predicted-values plot ideally show?

Random scatter around zero with no systematic pattern.

What does random scatter around zero in a residual plot suggest?

Residuals are independent of predicted values.

What does a systematic pattern in residuals versus predicted values suggest?

Possible model misspecification or nonlinearity.

What does increasing spread in a residual plot suggest?

Nonconstant variance.

What does decreasing spread in a residual plot suggest?

Nonconstant variance.

What does a funnel shape in residuals suggest?

Heteroskedasticity.

What should a residuals-versus-independent-variable plot ideally show?

Random scatter around zero with no directional pattern.

What does a directional relationship between residuals and an independent variable suggest?

The model may be missing a nonlinear term or may be misspecified.

What does it mean if residuals are scattered around zero across values of the independent variables?

Residuals are unrelated to the independent variables.

What is a Normal Q-Q plot used to assess?

Whether residuals are normally distributed.

What should a Normal Q-Q plot look like if residuals are normally distributed?

Points should fall approximately along a straight diagonal line.

What does deviation from the diagonal line in a Q-Q plot suggest?

Residuals may not be normally distributed.

What does a fat-tailed residual distribution look like in a Q-Q plot?

More observations appear far from the diagonal line in the tails.

What does right skewness look like in a Q-Q plot?

Observations in the right tail are above the theoretical distribution line.

What does left skewness look like in a Q-Q plot?

Observations in the left tail deviate below the theoretical distribution line.

In a standard normal distribution, approximately what percent of observations should be below −1.65 standard deviations?

5%.

If many observations fall beyond ±2 standard deviations, what does that suggest?

The residuals may have fat tails or outliers.

If there is an outlier beyond −3 standard deviations, what does that suggest?

A potentially extreme residual observation.

What is the formula for predicted value in multiple regression?

Ŷᵢ = b̂₀ + b̂₁X₁ᵢ + b̂₂X₂ᵢ + ... + b̂ₖXₖᵢ.

What is the formula for residual error?

ε̂ᵢ = Yᵢ − Ŷᵢ.

What is the formula for the regression model with two independent variables?

Y = b₀ + b₁X₁ + b₂X₂ + ε.

What is the formula for the estimated regression with two independent variables?

Ŷ = b̂₀ + b̂₁X₁ + b̂₂X₂.

What is the formula minimized by least squares regression?

Σᵢ₌₁ⁿ εᵢ².

What rule should be used for p-values?

Reject if p-value < significance level; do not reject if p-value > significance level.

What is the key interpretation rule for slope coefficients in multiple regression?

One-unit change in one variable changes the expected dependent variable by the coefficient amount, holding other variables constant.

What is the key interpretation rule for the intercept?

Expected dependent variable value when all independent variables equal zero.

What is the key residual plot rule?

Random scatter around zero is desirable; patterns suggest assumption violations.

What is the key Q-Q plot rule?

Points near the diagonal support normality; systematic deviations suggest nonnormality.

What is regression model specification?

Selection of explanatory variables and any transformations of those variables.

What should the chosen independent variables have?

Economic rationale.

What does it mean for a model to be parsimonious?

Simple and efficient.

What should a properly specified model do outside the sample?

Perform well.

What should a properly specified model not violate?

Key regression assumptions.

What are common forms of functional form misspecification?

Omitted variables, inappropriate variable form, inappropriate variable scaling, and improperly pooled data.

What is omitted variable misspecification?

Leaving out one or more variables that should be included based on economic theory.

What is the effect of omitting an important independent variable?

Biased and inconsistent regression parameters.

What residual problems can omitted variables cause?

Serial correlation or heteroskedasticity.

If an omitted variable is correlated with included independent variables, what happens?

The error term becomes correlated with those included variables.

If the error term is correlated with independent variables, what happens to coefficient estimates?

They become biased and inconsistent.

If an omitted variable is uncorrelated with the included independent variables, what is biased?

The intercept.

If an omitted variable is uncorrelated with the included independent variables, what happens to slope estimates?

They can still be correct.

What does omission mean in model specification?

A relevant variable is excluded from the model.

Is leaving out a highly correlated variable always an omitted variable problem?

No.

When might leaving out a variable be acceptable?

When the variable is highly correlated with another included variable and creates multicollinearity concerns.