3.2

Chapter 3: Describing Relationships

Section 3.2: Least-Squares Regression

Learning Targets

  • By the end of this section, you should be able to:

    • Make predictions using regression lines, keeping in mind the dangers of extrapolation.

    • Calculate and interpret a residual.

    • Interpret the slope and y-intercept of a regression line.

    • Determine the equation of a least-squares regression line using technology or computer output.

    • Construct and interpret residual plots to assess whether a regression model is appropriate.

    • Interpret the standard deviation of the residuals and r2r^2 and use these values to assess how well a least-squares regression line models the relationship between two variables.

    • Describe how the least-squares regression line, standard deviation of the residuals, and r2r^2 are influenced by outliers.

    • Find the slope and y-intercept of the least-squares regression line from the means and standard deviations of xx and yy and their correlation.

Regression Lines

  • Linear (straight-line) relationships between two quantitative variables are common.

  • A regression line summarizes the relationship between two variables only in a specific setting: when one variable helps explain the other.

  • Regression line equation: (\hat{y} = b0 + b1 x), where:

    • (\hat{y}) is the predicted value of yy for a given value of xx.

Prediction Example

  • A random sample of 16 used Ford F-150 SuperCrew 4×4s selected from autotrader.com has the regression equation:

    • (\hat{\text{price}} = 38257 - 0.1629 \times \text{miles driven})

  • Example: Predict the price of a Ford F-150 that has been driven 100,000 miles:

    • (\hat{\text{price}} = 38257 - 0.1629 \times 100000 = 21,967)

Extrapolation

  • Extrapolation is predicting values outside the range of data used to create the regression model.

  • Caution: predictions far outside this interval can be inaccurate.

  • Example: Predicting price for a Ford F-150 with 300,000 miles:

    • (\hat{\text{price}} = 38257 - 0.1629 \times 300000 = -10613) (nonsensical result, indicating extrapolation error).

Residuals

  • A residual is the difference between the actual value of yy and the predicted value of yy:

    • Residual = Actual yy - Predicted yy = y−y^y - \hat{y}.

  • In practice, no line will pass through all points; residuals measure prediction errors in yy.

  • Residual example using Ford F-150 driven 70,583 miles:

    • Find predicted price:

    • (\hat{\text{price}} = 38257 - 0.1629 \times 70583 = 26759)

    • If the actual price is 21,994,then:</p></li><li><p>Residual=21,994−26,759=−4765.</p></li></ul></li></ul><h5id="8e4aedb8−aeee−4603−99fc−f51e70a17f18"data−toc−id="8e4aedb8−aeee−4603−99fc−f51e70a17f18"collapsed="false"seolevelmigrated="true">InterpretingaRegressionLine</h5><ul><li><p>Aregressionlineisamodelanalogoustodensitycurves.</p></li><li><p>Componentsofregressionequation(y^=b<em>0+b</em>1x):</p><ul><li><p><strong>21,994, then:</p></li><li><p>Residual = 21,994 - 26,759 = -4765.</p></li></ul></li></ul><h5 id="8e4aedb8-aeee-4603-99fc-f51e70a17f18" data-toc-id="8e4aedb8-aeee-4603-99fc-f51e70a17f18" collapsed="false" seolevelmigrated="true">Interpreting a Regression Line</h5><ul><li><p>A regression line is a model analogous to density curves.</p></li><li><p>Components of regression equation (\hat{y} = b<em>0 + b</em>1 x):</p><ul><li><p><strong>b_0$: y-intercept (predicted $y$ when $x$ = 0).

    • $b_1$: slope (amount $y$ changes with a 1 unit increase in $x).</p></li></ul></li><li><p>ExampleInterpretation:ForFordF−150,<strong>).</p></li></ul></li><li><p>Example Interpretation: For Ford F-150, <strong>b_1 = -0.1629$ (the predicted price decreases by $0.1629 for each additional mile driven).

    • The value of b0=38257b_0 = 38257 gives a meaningful prediction for mileage near 0.

    The Least-Squares Regression Line

    • The least-squares regression line minimizes the sum of squared residuals, achieving the best fit.

    Determining Appropriateness of Linear Models
    Residual Plots

    • To assess a regression model's appropriateness, look for patterns in residual plots (vertical axis displays residuals, horizontal displays the explanatory variable).

    • Characteristics of a good fit:

      • No clear patterns in the residual plot.

      • Residuals should be relatively small in size.

    Evaluating Fit with ss and r2r^2

    • The standard deviation of residuals (s) measures the size of a typical residual, indicating average prediction error:

      • (s = \sqrt{\frac{\sum (y_i - \hat{y})^2}{n-2}})

    • The coefficient of determination (r^2) measures the percentage of variability in the response variable explained by the regression line:

      • (r^2 = 1 - \frac{\sum (\text{residuals})^2}{\sum (y_i - \bar{y})^2})

    Interpreting Technology Output

    • Interpret computer regression output, identifying:

      • Slope (b_1).

      • Y-intercept (b_0).

      • Value of standard deviation of residuals (s).

      • Value of coefficient of determination (r^2).

    Calculating the Regression Equation from Summary Statistics

    • Approach: Calculate means (\bar{x}), (\bar{y}), standard deviations s<em>xs<em>x, s</em>ys</em>y, and correlation rr to find:

      • Slope: (b1 = r \cdot \frac{sy}{s_x}).

      • Intercept: (b0 = \bar{y} - b1 \cdot \bar{x}).

    Regression to the Mean

    • If the explanatory variable xx increases by 1 standard deviation, the response variable yy increases by rr standard deviations: this is called regression to the mean.

    Correlation and Regression Wisdom

    • Correlation and regression are powerful but limited tools. Key considerations:

      • They describe only linear relationships.

      • Correlation does not imply causation; ensure understanding of variable relationships and causation implications.