Comprehensive Study Guide to Scatterplots, Least-Squares Regression Lines, and Residual Analysis

Core Principles of Least-Squares Regression and Scatterplots

• Definitions of Regression Variables:

  • Explanatory Variable ($x$): The independent variable that is hypothesized to explain, influence, or predict changes in the outcome variable. Plotted along the horizontal axis ($x$-axis).
  • Response Variable ($y$): The dependent variable measuring the outcome or result of interest. Plotted along the vertical axis ($y$-axis).
  • Predicted Value ($\hat{y}$): The estimated value of the response variable obtained by substituting a given value of $x$ into the regression model equation.

• Mathematical Form of the Least-Squares Regression Line (LSRL):

  • Standard Equation:     $\hat{y} = a + bx$
  • Slope ($b$): Represents the estimated rate of change in the predicted response variable ($\hat{y}$) for each unit increase in the explanatory variable ($x$).
  • $y$-Intercept ($a$): Represents the predicted value of the response variable ($\hat{y}$) when the explanatory variable ($x$) equals $0$.
  • Purpose of the LSRL: Provides a mathematically optimized linear model minimizing the sum of squared vertical deviations (residuals) between observed points and the line, allowing systematic prediction of the response variable from the explanatory variable.

• Correlation and Explained Variation Metrics:

  • Correlation Coefficient ($r$): Measures the strength and direction of the linear relationship between two quantitative variables. Bound between $-1$ and $+1$:
  • Positive correlation ($r > 0$): High values of $x$ tend to associate with high values of $y$.
  • Negative correlation ($r < 0$): High values of $x$ tend to associate with low values of $y$.
  • Near $0$: Little to no linear association.
  • Coefficient of Determination ($r^2$): The proportion (or percentage) of the total variation in the response variable ($y$) that is explained by the linear relationship with the explanatory variable ($x$).

• Residual Diagnostics and Evaluation Criteria:

  • Residual Calculation:     $\text{Residual} = y - \hat{y}$     where $y$ is the actual observed value and $\hat{y}$ is the predicted value from the regression model.
  • Overprediction vs. Underprediction:
  • If $\text{Residual} > 0$ (observed $>$ predicted): The linear model underpredicted the observed outcome.
  • If $\text{Residual} < 0$ (observed $<$ predicted): The linear model overpredicted the observed outcome.
  • Residual Plot Assessment: A plot of residuals versus explanatory values (or fitted values). If the residual plot exhibits random scatter with no discernible pattern, curvature, or fanning, the linear regression model is appropriate and represents the best fit for the data.
  • Interpolation vs. Extrapolation:
  • Interpolation: Making predictions within the domain of observed explanatory data values ($\min(x) \le x \le \max(x)$). Predictions made via interpolation carry high confidence.
  • Extrapolation: Making predictions outside the observed domain of $x$. Extrapolations carry low confidence and are considered unreliable because relationships may change beyond observed boundaries.

Training Duration Over Time Analysis

• Empirical Data Table:

  • Week $1$: $43\,\text{min}$
  • Week $2$: $39\,\text{min}$
  • Week $3$: $35\,\text{min}$
  • Week $4$: $33\,\text{min}$
  • Week $5$: $32\,\text{min}$
  • Week $6$: $30\,\text{min}$
  • Week $7$: $30\,\text{min}$
  • Week $8$: $28\,\text{min}$
  • Week $9$: $26\,\text{min}$

• Least-Squares Regression Equations:

  • Algebraic formula:     $\hat{y} = 42.39 - 1.9x$
  • Contextual formula:     $\text{Time} = 42.39 - 1.9(\text{week})$

• Interpretation of Model Parameters:

  • Slope ($b = -1.9$): For every increase of $1$ in week, there is an estimated decrease of $1.9\,\text{min}$ in time (expressed in the raw calculation as a change of $-1.9\,\text{min}$).
  • $y$-Intercept ($a = 42.39$): When the week is $0$, the predicted completion time is $42.39\,\text{min}$.

• Correlation and Variance Explained:

  • Correlation Coefficient ($r = -0.96$): Indicates a strong negative linear correlation between the progression of weeks and the required time.
  • Coefficient of Determination ($r^2 = 0.93$): Approximately $93\%$ ($0.93$) of the variation in time during each of the first $9$ weeks is explained by the variation in weeks.

• Goodness of Fit Diagnostic:

  • The residual plot displays no distinct pattern, demonstrating that a linear regression line is the best fit for the data.

Sports Analytics: Home Attendance Predicted by Wins

• Computer Regression Output Structure:

  • Predictor Constant: Coefficient = $-14364$, Standard Error of Coefficient ($\text{SE Coef}$) = $13428$, $t$-statistic = $-1.07$, $p$-value = $0.306$
  • Predictor Wins: Coefficient = $538.9$, Standard Error of Coefficient ($\text{SE Coef}$) = $160.2$, $t$-statistic = $3.36$, $p$-value = $0.006$
  • Standard Error of the Regression: $S = 6932.67$
  • Model Fit Statistics: $R\text{-Sq} = 48.5\%$, $R\text{-Sq(adj)} = 44.2\%$

• Mathematical and Contextual Regression Models:

  • Mathematical model:     $\hat{y} = -14364 + 538.9x$
  • Contextual model:     $\text{Home attendance} = -14364 + 538.9(\text{wins})$

• Interpretations of Parameters:

  • Slope ($b = 538.9$): For every increase of $1$ win, there is an estimated increase of $538.9$ attendees in home attendance.
  • $y$-Intercept ($a = -14364$): When the number of wins is $0$, the predicted home attendance is $-14364$.

• Correlation and Determination Measures:

  • Coefficient of Determination ($r^2 = 48.5\%$): Approximately $48.5\%$ of the variation in home attendance is accounted for by the variation in the number of wins.
  • Correlation Coefficient ($r = 0.696$): Indicates a moderate positive linear correlation between the number of wins and home game attendance.

Geological Stratigraphy: Depth and Montmorillonite Percentage

• Field Data Observations:

  • Depth $40\,\text{m}$: $58\%$ montmorillonite
  • Depth $50\,\text{m}$: $34\%$ montmorillonite
  • Depth $60\,\text{m}$: $32\%$ montmorillonite
  • Depth $70\,\text{m}$: $30\%$ montmorillonite
  • Depth $80\,\text{m}$: $28\%$ montmorillonite
  • Depth $90\,\text{m}$: $27\%$ montmorillonite
  • Depth $100\,\text{m}$: $22\%$ montmorillonite

• LSRL Formulation and Parameter Interpretation:

  • Regression equation:     $\hat{y} = 64.5 - 0.45x$
  • Slope ($b = -0.45$): For every increase of $1$ unit in depth, there is an estimated decrease of $0.45\%$ in montmorillonite clay content.

• Correlation, Determination, and Outlier Impact:

  • Correlation Coefficient ($r = -0.83$): The association of the linear regression is negative, showing a strong negative correlation between depth and montmorillonite percentage.
  • Coefficient of Determination ($r^2 = 0.6931$ or $69.31\%$): $69.31\%$ of the variation in montmorillonite percentage is accounted for by the variation in depth.
  • Impact of Outliers: Removing the single outlier (at depth $40$, value $58\%$) substantially strengthened the negative linear correlation from $r = -0.849$ to $r = -0.976$.

• Model Diagnostic:

  • There is no discernible pattern in the residual plot, confirming that the linear regression is the best fit for the dataset.

Ecological Dynamics: Wildebeest Density and Burned Grassland Extent

• Computer Regression Output Structure:

  • Predictor Constant: Coefficient = $92.29$, Standard Error of Coefficient ($\text{SE Coef}$) = $10.06$, $t$-statistic = $9.17$, $p$-value = $0.000$
  • Predictor Wildebeest ($1000\text{s}$): Coefficient = $-0.05762$, Standard Error of Coefficient ($\text{SE Coef}$) = $0.01035$, $t$-statistic = $-5.56$, $p$-value = $0.000$
  • Residual Variation: $S = 15.9880$
  • Coefficient Metrics: $R\text{-Sq} = 64.1\%$, $R\text{-Sq(adj)} = 62.5\%$

• Regression Modeling:

  • Equation:     $\hat{y} = 92.29 - 0.058x$
  • Variable specification: $x = \text{wildebeest population (in thousands)}$, $\hat{y} = \text{predicted percent of grass area burned}$.

• Parameter and Metric Interpretations:

  • Slope ($b = -0.058$): For every increase of $1$ unit ($1000$ animals) in the wildebeest population, there is an estimated decrease of $0.058\%$ in the percentage of grass area burned.
  • Correlation Coefficient ($r = -0.804$): Indicates a strong negative linear correlation between wildebeest density and the percentage of grass area burned.

• Residual Evaluation:

  • Because there is no discernible systematic pattern in the residual plot, a linear model is confirmed to be appropriate for this dataset.

Workplace Compensation: Years of Experience vs. Yearly Income

• Employee Profile Dataset ($12$ Observations):

  • Employee #$1$: $(22, 95)$
  • Employee #$2$: $(24, 100)$
  • Employee #$3$: $(18, 35)$
  • Employee #$4$: $(10, 55)$
  • Employee #$5$: $(7, 50)$
  • Employee #$6$: $(4, 85)$
  • Employee #$7$: $(2, 35)$
  • Employee #$8$: $(7, 63)$
  • Employee #$9$: $(12, 74)$
  • Employee #$10$: $(8, 40)$
  • Employee #$11$: $(15, 72)$
  • Employee #$12$: $(1, 65)$

• Tabular Organization of Paired Data:

  • Experience ($x$, in years): $22, 24, 18, 10, 7, 4, 2, 7, 12, 8, 15, 1$
  • Income ($y$, in thousands of dollars): $95, 100, 35, 55, 50, 85, 35, 63, 74, 40, 72, 65$

• Variable Designation:

  • Explanatory Variable ($x$): Number of years of experience.
  • Response Variable ($y$): Yearly income earned (in thousands of dollars).

• Regression Line Formulation:

  • Mathematical LSRL equation:     $\hat{y} = 48.67 + 1.42x$
  • Contextual LSRL equation:     $\text{Yearly income in thousands of dollars} = 48.67 + 1.42(\text{number of years})$

• Parameter and Functional Purpose Interpretations:

  • Slope Interpretation ($b = 1.42$): As the number of years of experience increases by $1$, the predicted yearly income in thousands of dollars will increase by approximately $1.42$ (equivalent to an increase of approximately $\$1420$).
  • Functional Purpose of the LSRL: Used to predict the yearly income (in thousands of dollars) earned by an employee based on their total years of work experience.

Academic Performance: Sleep Duration and Exam Scores

• Exam and Sleep Dataset:

  • Hours Slept ($x$): $8, 7, 7, 8, 7, 7, 8, 6, 5, 7, 4, 9, 7$
  • Test Score ($y$): Paired values corresponding to observed student scores including $83, 86, 74, 88, 76, 63, 90, 60, 89, 81$, with student Mary recording $6\,\text{hours}$ of sleep and an exam score of $76$.

• LSRL Modeling:

  • Mathematical equation:     $\hat{y} = 35.63 + 6.38x$
  • Contextual equation:     $\text{Test Score} = 35.63 + 6.38(\text{Hours slept})$

• Parameter and Determination Analysis:

  • Slope Interpretation ($b = 6.38$): For every increase of $1\,\text{hour}$ of sleep the night before the test, the predicted test score increases by $6.38$ points.
  • Coefficient of Determination ($r^2 = 78\%$): $78\%$ of the variation in test scores is accounted for by the variation in hours slept the night before the exam.
  • Linear Fit Diagnostic: The residual plot exhibits no systematic pattern, confirming that the linear regression is the best fit for this data.

• Prediction and Extrapolation Risk ($x = 3$):

  • Model calculation:     $\hat{y} = 35.63 + 6.38(3) = 35.63 + 19.14 = 54.77$
  • Confidence level and reasoning: Very low confidence in this predicted test score because $x = 3$ constitutes extrapolation, falling strictly outside the domain of observed predictor data ($4$ to $9\,\text{hours}$).

• Residual Computation for Mary ($x = 6$, $y = 76$):

  • Model prediction:     $\hat{y} = 35.63 + 6.38(6) = 35.63 + 38.28 = 73.91$     (recorded in working notes as $73.96$
  • Residual calculation:     $\text{Residual} = 76 - 73.96 = 2.04$
  • Residual interpretation: The regression model underpredicted Mary's actual exam performance by $2.04$ units.

Environmental Impact on Sales: Daily Temperature and Vending Machine Purchases

• Empirical Observations ($15$ Sample Days):

  • Daily High Temperature ($x$, in $^\circ\text{F}$): $70, 75, 80, 90, 93, 98, 72, 75, 75, 80, 90, 95, 98, 91, 98$
  • Soft Drink Quantity Sold ($y$, in cans): $30, 31, 40, 52, 57, 59, 33, 38, 32, 45, 53, 56, 62, 51, 58$

• Variable Classification:

  • Explanatory Variable ($x$): Temperature ($^\circ\text{F}$).
  • Response Variable ($y$): Quantity of soft drink cans sold.

• Model Equations:

  • Standard mathematical formulation:     $\hat{y} = -46.17 + 1.09x$
  • Contextual formulation:     $\text{Quantity} = -46.17 + 1.09(\text{Temperature})$

• Model Validity and Correlation Analysis:

  • Linear Appropriateness: Linear regression is appropriate for this dataset because the residual plot contains no discernible curved or clustered pattern.
  • Correlation Coefficient ($r = 0.98$): Indicates an extremely strong positive linear relationship between daily ambient temperature and the quantity of soft drink cans sold.
  • Coefficient of Determination ($r^2 = 0.9583$ or $95.83\%$): Approximately $95.83\%$ of the total variation in the quantity of soft drink cans sold is directly explained by the linear relationship with daily temperature.

• Interpretations of Intercept and Slope:

  • Slope ($b = 1.09$): For every $1^\circ\text{F}$ increase in daily temperature, the predicted quantity of soft drinks sold increases by $1.09$ cans.
  • $y$-Intercept ($a = -46.17$): When the daily temperature is theoretical absolute zero degrees ($0^\circ\text{F}$), the predicted quantity of soft drinks sold is $-46.17$ cans.

• Predictive Performance and Boundary Evaluations:

  • Prediction at $T = 85^\circ\text{F}$:     $\hat{y} = -46.17 + 1.09(85) = -46.17 + 92.65 = 46.48\,\text{cans}$
  • Reasonableness: Highly reasonable because $85^\circ\text{F}$ represents interpolation, falling safely within the observed explanatory data bounds ($70^\circ\text{F} \le 85^\circ\text{F} \le 98^\circ\text{F}$).
  • Prediction at $T = 62^\circ\text{F}$:     $\hat{y} = -46.17 + 1.09(62) = -46.17 + 67.58 = 21.41\,\text{cans}$
  • Reasonableness: Not reasonable because $62^\circ\text{F}$ represents extrapolation, falling below the minimum observed sample temperature ($70^\circ\text{F}$).

• Specific Residual Calculations and Diagnostic Interpretations:

  • Evaluation at $x = 72^\circ\text{F}$ ($y = 33$):     $\hat{y} = -46.17 + 1.09(72) = -46.17 + 78.48 = 32.31$$\text{Residual} = 33 - 32.31 = 0.69$
  • Interpretation: The regression model underpredicted soft drink sales by $0.69$ cans.
  • Evaluation at $x = 91^\circ\text{F}$ ($y = 51$):     $\hat{y} = -46.17 + 1.09(91) = -46.17 + 99.19 = 53.02$$\text{Residual} = 51 - 53.02 = -2.02$
  • Interpretation: The regression model overpredicted soft drink sales by $2.02$ cans.