Chapter 5 - Regression

Chapter 5: Regression Notes

Overview of Regression

  • Regression Line: A straight line displaying how a response variable (y) changes in response to an explanatory variable (x).
  • Prediction Objective: Typically used to predict values of y from known values of x, assuming a linear relationship.
  • Example: If nonexercise activity (NEA) changes by 400 calories, estimate the corresponding fat gain.

Understanding Regression Lines

  • General Form of Equation: The regression equation is expressed as: [ y = a + bx ] where:
    • y: Response variable
    • x: Explanatory variable
    • b: Slope (indicating how much y changes per unit change in x)
    • a: Intercept (the value of y when x = 0)
  • Plotting Lines: To visualize, use predicted y values for different x values to draw the regression line.

Least-Squares Regression Line

  • Definition: The least-squares regression line minimizes the sum of the squares of the vertical distances (residuals) between actual data points and the predicted values from the line.
  • Equation of Least-Squares Regression: [ y = a + bx ] where:
    • Slope (b): Calculated using the means and standard deviations of both variables and their correlation.
    • Intercept (a): Calculated using the means of both variables.
Example Calculation
  • Regression Example: For NEA change, the equation is
    [ ext{fat gain} = 3.505 - 0.0034 imes ext{(NEA in calories)} ]
  • Prediction: For 400 calories of NEA:
    [ ext{Fat gain} = 3.505 - 0.0034 imes 400 = 2.13 ext{ kg} ]

Key Facts about Least-Squares Regression

  • Variable Distinction: Understand the difference between explanatory and response variables is crucial.
  • Slope Relation to Correlation: The slope of the regression line is related to the correlation coefficient; both have the same sign.
  • Standard Deviation Change: A change of 1 standard deviation in x results in a change of ( r ) standard deviations in y.
  • Line Intercept: The least-squares line always passes through the point (mean of x, mean of y).
  • Utility: R-squared (( r^2 )): Represents the proportion of variation in y explained by x.

Residuals

  • Definition: Residuals are differences between observed values and predicted values from the regression line:
    [ ext{Residual} = ext{Observed} - ext{Predicted} ]
  • Properties: Mean of residuals is always zero.
  • Residual Plots: Used to assess the fit of regression line, with a good fit indicated by random scatter around zero.

Influential Observations

  • Definition: An observation is influential if its removal significantly impacts the regression outcome.
  • Outliers: Points that deviate from the trends can skew results and should be examined critically.

Cautions in Correlation and Regression

  • Linear Relationships Only: Correlation and regression lines solely describe linear associations.
  • Vulnerability to Outliers: Outliers can heavily influence calculations.
  • Beware of Extrapolation: Making predictions outside the data range can lead to erroneous conclusions.
  • Lurking Variables: Factors that are not included in the study can influence outcomes.
  • Causation vs Correlation: Correlation does not imply causation; strong relationships may not indicate genuine causative effects.
Evidence of Causation
  • Experimental Design: Randomized experiments provide the most reliable evidence for causation.
  • Observational Correlation: Strong, consistent associations, the timing of cause and effect, and plausible connections can support causal claims in non-experimental settings.

Big Data Considerations

  • Definition: Big data refers to massive datasets collected for analysis by organizations.
  • Claims of Big Data: Big data can predict outcomes without worrying about causation, but pitfalls include bias and misinterpretation of correlation.
  • Need for Theory: Understanding and theory remain vital; big data insights should not replace statistical evidence and careful analysis.