Examining Relationships: Scatterplots, Correlation, and Regression

Response Variable ( $Y$ ): Measures the primary outcome of a study.
Explanatory Variable ( $X$ ): Explains, influences, or predicts changes in the response variable.
Lurking Variable: A variable not included in the study that influences the interpretation of relationships among variables.
Arbitrary Assignment: When studying the mutual relationship between two variables without a predictive direction, designating which is $X$ and which is $Y$ is arbitrary.

Purpose: Displays the relationship between two quantitative variables measured on the same set of individuals.
Axis Convention: The explanatory variable is plotted on the $x$ -axis and the response variable is plotted on the $y$ -axis.
Key Characteristics to Assess:
- Direction: Positive association (upward slope from left to right) or negative association (downward slope from left to right).
- Form: Overall shape of the pattern (linear, quadratic, exponential, logarithmic, or non-linear).
- Strength: Proximity of points to the underlying form (strong vs. weak scatter).
- Outliers: Individual values falling outside the overall pattern of the rest of the data.
Categorical Subgroups: Apparent non-linear or clustered distributions may reflect two or more distinct underlying populations (such as separate subgroups for males and females).

Definition: A standardized measure of the direction and strength of a linear relationship between two quantitative variables.
Formula: $r = \frac{1}{(n - 1)s_x s_y} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})$
Key Properties:
- Bounded Range: $-1 \le r \le 1$ .
- Sign Interpretation: Positive values indicate positive linear associations; negative values indicate negative linear associations.
- Extremes: $r = 1$ or $r = -1$ occurs strictly for perfect straight-line relationships.
- No Units: Standardizing deviations by standard deviations renders $r$ scale-free and unaffected by changes in measurement units.
- Symmetry: Switching the roles of $X$ and $Y$ does not change $r$ .
- Linearity Limitation: Measures strictly linear associations; non-linear relationships can have $r \approx 0$ despite strong structural associations.
- Sensitivity: Highly non-resistant to outliers.
Coefficient of Determination ( $r^2$ ): Represents the proportion (or percentage) of total variation in $Y$ accounted for by its linear regression on $X$ .

Model Equation: $\hat{y} = b_0 + b_1 x$
- $\hat{y}$ represents the predicted response value for a given explanatory value $x$ .
- $b_1$ represents the slope.
- $b_0$ represents the $y$ -intercept.
Slope ( $b_1$ ): $b_1 = r \frac{s_y}{s_x}$
- Represents the predicted change in $y$ when $x$ increases by one unit.
- Carries the same sign as the correlation coefficient $r$ .
Intercept ( $b_0$ ): $b_0 = \bar{y} - b_1 \bar{x}$
- Represents the predicted value of $y$ when $x = 0$ .
Criterion: Minimizes the sum of squared vertical prediction errors: $\sum_{i=1}^n (y_i - \hat{y}_i)^2$
Centroid Point: The least squares regression line always passes through the point of means, $(\bar{x}, \bar{y})$ .

Vertical deviation between actual and predicted response values

Residual Definition: The vertical difference between an observed data value and its predicted value: $\text{residual}_i = y_i - \hat{y}_i$
Sign Interpretation: A positive residual represents a point positioned above the regression line; a negative residual indicates a point below the line.
Extrapolation: The practice of predicting response values for $x$ -values outside the range of observed data. It is unreliable because the linear pattern may not hold outside the observed interval.

Comparison of regression lines with and without an influential outlier

Outliers in the $y$ -Direction: Observations exhibiting large vertical residuals; typically have moderate to small leverage on the slope of the fitted line.
Outliers in the $x$ -Direction (High Leverage): Observations extreme in the horizontal dimension; strongly pull the regression line toward themselves.
Influential Observation: An individual data point whose removal causes a substantial change in the regression parameters ( $b_0$ , $b_1$ ) or $r^2$ .

Core Principle: Association does not imply causation. A strong correlation between $X$ and $Y$ does not establish that changes in $X$ cause changes in $Y$ .
Observational Studies: Variables are observed without experimenter intervention, leaving observed associations susceptible to confounding by lurking variables.
Experimental Studies: Explanatory variables are actively and randomly assigned to units, balancing out lurking variables across treatment groups to provide valid evidence of causation.