Linear Regression, Correlation, Outliers, and Extrapolation

Association Versus Causation in Linear Correlation

• Correlation does not equal causation: a strong linear relationship between two variables provides evidence of association, but never proves that one variable causes changes in the other.

• Regression analysis and correlation analysis quantify the strength and direction of linear association; they cannot be used to conclude that an event occurred because of another variable.

• Case Study: Nuclear Tests Versus Birth Rates:

  • An observed scatter plot demonstrates a strong negative linear correlation between the number of nuclear tests conducted and the birth rate.
  • As the number of nuclear tests increases, the number of births decreases.
  • The estimated correlation coefficient for this relationship is $r = -0.95$.
  • Despite the strong linear relationship ($r = -0.95$), it is mathematically incorrect to claim that nuclear testing causes fewer births or that the decline in birth rates happens because of the testing.

• Lurking Variables (Hidden Variables):

  • A lurking variable is an unmeasured or hidden variable not included among the observed variables that influences the relationship between them.
  • Apparent associations often stem from lurking variables rather than a direct causal mechanism between the analyzed variables.

• Misuse in Media and Advertising:

  • Scatter plots showing strong correlation are frequently used in advertisements, media, and news broadcasts to create a false impression of cause-and-effect relationships.

• Examination Framework:

  • Multiple-choice questions frequently test whether an observed correlation can justify a causal claim; correlation must strictly be identified as association rather than cause-and-effect.

Sensitivity of the Correlation Coefficient to Outliers

• Diagnostic Requirement:

  • A scatter plot must always be constructed and visually inspected prior to performing linear regression or calculating the correlation coefficient ($r$).

• Outlier Sensitivity of $r$:

  • The linear correlation coefficient $r$ is highly sensitive to outliers.
  • The presence of even a single outlier can dramatically distort the value of $r$, rendering it completely unreliable.

• Case Study: Study Hours Versus Exam Scores:

  • A baseline dataset compares study time (in hours) to exam scores evaluated out of $10$.
  • The initial scatter plot displays a very strong positive linear correlation (greater study duration corresponds to higher grades), with an estimated correlation coefficient of $r = 0.96$.
  • An outlier is added to the dataset: an individual who studied for $12\text{ hours}$ and earned an exam score of $0$.
  • Upon including this single outlier, the correlation coefficient collapses from approximately $r = 0.96$ down to $r = 0.01$.

• Investigating Causes of Outliers:

  • When an outlier is detected in a dataset, the first possibility to evaluate is data recording error (for example, a data entry mistake where an exam score intended to be entered as $10$ was entered as $0$).

• Influence of Sample Size:

  • In small datasets, a single outlier exerts a massive distorting effect on $r$.
  • In larger datasets, a single point produces less numerical variance on the metric, but $r$ remains fundamentally vulnerable to extreme values.

Inappropriate Modeling: Fitting Linear Models to Nonlinear Data

• Nonlinearity Pitfalls:

  • A common methodological error is attempting to fit a linear model to data that exhibits a nonlinear pattern.
  • A linear regression line cannot accurately model or represent nonlinear data distributions.

• Modeling Criteria:

  • A single linear slope cannot describe curved or nonlinear trends.
  • A scatter plot must be evaluated visually first to verify that the underlying relationship is linear before attempting to determine a best-fit line.

The Hazards of Extrapolation Beyond the Range of Data

• Principle of Extrapolation:

  • Extrapolation is the practice of predicting response values for explanatory variables that fall outside the observed range (domain) of the data.
  • Predictions must never be made beyond the range of collected data because such extrapolations are inherently unreliable.

• Case Study: Age Versus Height:

  • A dataset tracks age and height, where the domain of age ranges strictly from $3$ to $20\text{ years}$.
  • The empirical data indicates that height increases from age $3$ up to approximately age $15$, after which human growth levels off through age $20$.
  • Attempting to predict the height of a person at age $25$ using a linear model fitted to the $3$ to $20$ age range yields an erroneous prediction of approximately $85\text{ inches}$.
  • Because human growth plateaus and does not continue linearly after adolescence, using a linear model outside its observed domain ($3$ to $20\text{ years}$) produces invalid and inaccurate predictions.

Questions & Discussion

• Question / Discussion: Scheduling the Chapter $4$ Quiz and class workflow:
  • Query: Should the Chapter $4$ quiz be held on Thursday, Monday, or Tuesday?
  • Student Response / Preference: Tuesday.
  • Instructor Resolution: The quiz is officially scheduled for Tuesday to avoid clustering assessments on Thursday.
  • Status of Material: Chapter $4$ is nearly complete, with only one remaining example left to cover.
  • Classroom Action: The class transitions immediately to completing Group Work $17$.