Sydney air quality example: Penrith vs Randwick

Topic: Linear Modeling and the Six-Step Regression Framework

• Context: End of module 2 on modeling; focus on linear regression, correlation, predictive modeling, and diagnostics using Sydney air quality data as a running example.
• Practical goal: Learn to go from data to a fitted linear model, check assumptions, and understand what the numbers and plots mean for real data.

Key Concepts and Definitions

• Bivariate data and the linear question

  • Two variables: x (predictor, independent) and y (response, dependent).
  • Core question: Is y linearly related to x? Does a line y_hat = a + b x fit well?

• Graphical and numerical summaries

  • Scatter plot: primary graphical summary of the relationship between x and y.
  • Correlation coefficient r: primary numerical summary of linear association.
  • Point of averages: the center of the cloud, given by
    $(\bar{x}, \bar{y})$
  • Two standard deviations (SDx, SDy) give a sense of spread along each axis.
  • r measures how tightly points cluster around the best-fit line; sign indicates direction of association; magnitude indicates strength.

• Key formulas

  • Correlation coefficient (sample):
    $r = \frac{ ext{Cov}(X,Y) }{ s<em>x \, s</em>y } = \frac{ \frac{1}{n}\, ext{sum}<em>{i=1}^n (x</em>i-\bar{x})(y_i-\bar{y}) }{ \frac{ ext{SD}(X) ext{SD}(Y) }{ } }$

  • Equivalent common form using sums:
    r = rac{ igl( extstylerac{1}{n}igr)
    extstyleigl(
    abla xi abla yiigr) }{ sx sy } = rac{ extstyleigl(ar{xy}-ar{x}ar{y}igr) }{ sx sy } ext{ (equivalents)}

  • Population correlation (conceptual):
    $r<em>{ ext{pop}} = \frac{1}{n} \, extstyle\bigl( z</em>{x,1} z<em>{y,1} + \nabla z</em>{x,2} z<em>{y,2} + \ ext{…} \bigr) ext{ with } z</em>{x,i} = \frac{x<em>i-\bar{x}}{s</em>x}, \, z<em>{y,i} = \frac{y</em>i-\bar{y}}{s_y}$

  • Linear regression line (prediction):
    $y<em>{ ext{hat}} = a + b x$ where $b = \frac{ ext{Cov}(X,Y)}{ ext{Var}(X) } = r \, \frac{s</em>y}{s<em>x}, \ a = \bar{y} - b \, \bar{x}, \ y</em>{ ext{hat}} = \bar{y} + r \, \frac{s<em>y}{s</em>x} \, (x - \bar{x}).$

  • Regression vs. standard deviation line

  • SD line: connects (xbar, ybar) to (xbar + sx, ybar + sy); slope = sy / sx.

  • Regression line: uses r; slope = r (sy / sx); passes through (xbar, ybar).

  • Residuals and residual plot

  • Residual: $e<em>i = y</em>i - ilde{y}<em>i$ where $ilde{y}</em>i$ is the predicted value from the line.

  • Residual plot: plot residuals e_i versus fitted values or x; diagnose linearity and homoskedasticity.

  • RMS error (predictive error summary)

  • For population:
    ext{RMS}_{E, ext{pop}} = ext{RMSE} = ext{E}ig[(Y - ilde{Y})^2ig]^{1/2} = ig( ext{Var}(Y|X)ig)^{1/2} ext{ under the model}
    

  • Resulting compact form when using r and sy: $ext{RMS}</em>{E, ext{pop}} = ext{sd}(Y) \, \big(1 - r^2\big)^{1/2} = ext{sd}_y \, \big(1 - r^2\big)^{1/2}$

  • Special cases:

  • If r = ±1, RMS error = 0 (perfect linear fit).

  • If r = 0, RMS error reduces to the baseline standard deviation sd(y) (the regression line carries no predictive power).

  • If homoscedasticity holds (constant spread across x), RMS error can be interpreted as spread within vertical strips around the line.

• Important cautions

  • High r does not guarantee a good linear fit (check residuals, not just r).
  • Anscombe's quartet: same five summaries (means, standard deviations, r) can yield different scatter plots; do not rely on r alone.
  • Ecological correlation: grouping or aggregating data can inflate apparent linear association.
  • Causation vs association: a linear relationship does not imply causation.

The Six-Step Linear Regression Framework

• Step 1: Produce a scatter plot to assess linearity and view potential outliers.

• Step 2: Compute the correlation coefficient r to gauge linear association.

• Step 3: Add the regression line (y_hat) to the scatter plot.

• Step 4: Produce a residual plot (residuals vs fitted values or x) to diagnose fit and assumptions.

• Step 5: Check assumptions implied by the model: linearity, independence, homoscedasticity, residual distribution.

• Step 6: Perform predictions using the model if Step 5 looks acceptable.

• Relationship among steps: you should not jump to Step 6 before verifying Steps 1–5; the regression line is a summary that depends on earlier steps.

• Practical notes about each step

  • Step 1: Scatter plot reveals potential nonlinear patterns (e.g., quadratic patterns) where a straight line is inappropriate.
  • Step 2: r quantifies linear association; remember its sign and magnitude; r alone is not enough for predictive adequacy.
  • Step 3: The regression line is the best linear predictor under least squares; it minimizes squared residuals along the line.
  • Step 4: The residual plot should show no systematic pattern and constant spread (homoscedasticity).
  • Step 5: If the residual plot shows pattern or heteroscedasticity, reconsider the model or transform the data.
  • Step 6: Predictions are valid only within the range of observed x-values (avoid extrapolation beyond the data without justification).

• Worked example: Sydney air quality data (Randwick vs Penrith, PM10)

  • Data: daily measurements for roughly a year; sites Randwick (eastern Sydney) and Penrith (western Sydney); pollutants include PM10, PM2.5, NO2, etc.; data from NSW government.
  • Research question (example): Does Penrith PM10 linearly relate to Randwick PM10? Can we predict Penrith PM10 from Randwick PM10?
  • Step 1: Scatter plot shows an apparent positive linear trend with some spread and an outlier.
  • Step 2: Correlation coefficient computed as $$ r \