Chapter 7: Linear Regression

Linear Model

A straight-line equation used to predict a response variable (y) based on an explanatory variable (x). It summarizes the general trend in a scatterplot.

Residual

The difference between an observed value and its predicted value.

Formula: Residual = Actual y – Predicted y

Residuals show how far off the model’s prediction is.

Least Squares Line

The line of best fit minimizes the sum of squared residuals. It gives the most accurate linear model for the data.

Correlation Coefficient (r)

Measures the strength and direction of the linear relationship.

• r = 1 or -1 means perfect linear relationship

• r = 0 means no linear relationship

• Closer to ±1 = stronger linear relationship

Regression to the Mean:

Predictions tend to be less extreme than the observed x-values. Predicted values are pulled closer to the mean.

Regression Line Equation

ŷ = b₀ + b₁x

Where:

• ŷ is the predicted y value

• b₁ is the slope (change in y per unit x)

• b₀ is the y-intercept (predicted y when x = 0)

Slope Interpretation

Describes the predicted change in y for each 1-unit increase in x.

Intercept Interpretation

Predicted value of y when x = 0. Sometimes it may not make sense in context.

Standard Error (s)

Measures the typical size of a residual. It tells how much the data points typically deviate from the regression line.

R-squared (R²)

Tells how well the regression model explains the variability in the response variable.

• R² = 0%: Model explains none of the variation

• R² = 100%: Model explains all the variation

• R² = (correlation)²

Conditions for Linear Regression

1. Quantitative Variables Condition: Both x and y must be quantitative.

2. Straight Enough Condition: The scatterplot should look roughly linear.

3. Outlier Condition: Outliers can distort the model. They must be checked.

4. Does the Plot Thicken? Condition: Residuals should have constant spread (equal variance).

Residual Plot

A graph of residuals vs. predicted values. Should look random (no pattern) if the model is appropriate.