Chapter 3: Linear Regression

Linear Regression

[Regression] This approach assumes a linear relationship between the predictors (X1, X2, ..., Xp) and the response variable (Y). The goal is to approximate the true regression function, even though true regression functions are rarely linear.

Simple Linear Regression

[Regression] This involves a single predictor X and the model Y = β0 + β1X + ε, where β0 and β1 are the intercept and slope, respectively, and ε is the error term. Future sales can be predicted using ŷ = β̂0 + β̂1x, where β̂0 and β̂1 are estimates for the model coefficients.

Multiple Linear Regression

[Regression] This extends simple linear regression to include multiple predictors, with the model Y = β0 + β1X1 + β2X2 + · · ·+ βpXp + ε. Here, βj represents the average effect on Y of a one-unit increase in Xj, holding all other predictors fixed.

Assessing Overall Model Accuracy

[Regression] Residual Standard Error (RSE) is calculated as RSE = √((1 / (n−2)) * RSS), where RSS is the residual sum-of-squares.

R-squared (R2)

represents the fraction of variance explained by the model and is calculated as R2 = (TSS − RSS) / TSS = 1 − (RSS / TSS), where TSS is the total sum of squares.

Variable Selection in Multiple Linear Regression

[Regression] To determine which predictors are useful, an F-statistic is used. Forward selection starts with a null model and adds variables, while backward selection starts with all variables and removes the least significant ones.

Qualitative Predictors

[Regression] Qualitative predictors with more than two levels require the creation of dummy variables. With k levels, k-1 dummy variables are created, and the level with no dummy variable is the baseline.

Interaction Effects

[Regression] These consider whether the effect of one predictor on the response depends on the value of another predictor. If an interaction term is included, the main effects should also be included, even if their p-values are not significant.

Non-linear Relationships

[Regression] Polynomial regression can be used to model non-linear effects.

Other Considerations

[Regression] Outliers, non-constant variance of error terms, high leverage points, and collinearity are important considerations in the regression model.