Reading 10: Simple Linear Regression

Book 1: Quantitative Methods

What is the purpose of simple linear regression (SLR)?

explain that variation in a DV in terms of the variation in the IV

What is the difference between variance and variation?

Variance – the average of the squared differences to the mean

Variation – the degree to which a variable differs from its mean value

Dependent Variable

the variable whose variation is explained by the IV

Names for the DV

explained variable

endogenous variable

predicted variable

Independent Variable

the variable used to explain the variation of the DV

Other names for the IV

Explanatory variable

Exogenous variable

Predicting variable

Where does the regression line pass through the y-intercept?

Where the coordinates are equal to the mean of the IV and DV…

When there is no variation in the IV, it shows the value of the DV

Regression Line

the line that minimizes the sum of the squared differences between the predicted y-values

Sum of Squared Errors (SSE)

the sum of squared vertical distances between the estimated and actual Y-values

What does the slope coefficient measure?

the expected change in the DV for every one-unit change in the IV

“The slope coefficient in a regression of the excess returns of an individual security compared to the market return is ______”

beta

What are the assumptions of a linear regression model?

1.) A linear relationship exists

2.) Homoskedasticity

3.) The residual term is independently distributed

4.) The residual term is normally distributed

What is Homoskedasticity?

the variance of the residual term is constant for all observations

Heteroskedasticity (2 versions)

when the variance of the residual term is not constant for all observations

OR

when the variance is the same but changes over time

Explain the assumption “the residual term is independently distributed”

assuming that the residual term is not correlated to all other residual terms

TRUE or FALSE:

If the relationship between X and Y are not independent, that does not necessarily mean that the residual terms are not independent.

FALSE

The residual terms are always not independent

What is ANOVA?

analyzes the total variability of the DV

• it analyzes the explained variance and the unexplained variance

Total Sum of Squares (SST)

measures the total variation of the DV

Sum of Squares Regression (SSR)

measures the explained variation of the DV by the IV

Mean Square Regression (MSR)

SSR divided by the # of IVs

Sum of Square Errors (SSE)

measures the unexplained variance in the DV

Mean Squared Error (MSE)

SSE divided by the degrees of freedom

Coefficient of Determination

the percentage of the total variation in the DV explained by the IV

What does a coefficient of determination of 0.63 mean?

“the variation in the IV explains 63% of the variation in the DV.”

For SLR, the coefficient of determination can be computed by…

(correlation coefficient)²

Is the F-Test a one- or two-tailed test?

One-tailed

For SLR, the F-Test is equivalent to…

the T-Test of the statistical significance of the slope coefficient

What do you perform a T-Test on?

the regression coefficient

What happens if the assumptions of a Linear Regression Model aren’t violated?

You need to transform the variables

Log-Lin Model

if the DV is logarithmic and the IV is linear

“slope coefficient is the relative change in the DV for an absolute change in the IV”

Lin-Log Model

if the DV is linear and the IV is logarithmic

“slope coefficient is the absolute change in the DV for an absolute change in the IV”

Log-Log Model

Both DV and IV are logarithmic

“Slope coefficient is interpreted as the relative change in the DV for a relative change in the IV”