Chapter 11: Multiple Regression and Correlation Flashcards

1. Introduction to Multivariate Relationships

  • Bivariate regression: relationship between two quantitative variables.

  • Multivariate regression: includes multiple explanatory variables to improve prediction and control for confounding.

  • Purpose: Identify whether associations change or disappear when controlling for other variables.

2. Multiple Regression Function

  • General form (two predictors):
    E(y) = α + β₁x₁ + β₂x₂

  • Each β coefficient = partial regression coefficient (effect of a predictor while holding others constant).

  • With more predictors: add more βx terms.

3. Interpretation of Parameters

  • α (intercept): Mean of y when all predictors = 0.

  • βᵢ: Change in mean of y for a one‑unit increase in xᵢ, controlling for all other predictors.

4. Example: Education & Crime in Florida

  • Bivariate model:
    E(y) = –51.3 + 1.5x₁ (crime ↑ as education ↑)

  • But education and urbanization are highly correlated → spurious correlation.

  • Multiple regression model:
    E(y) = 58.9 – 0.6x₁ + 0.7x₂

  • Controlling for urbanization reverses the effect → Simpson’s Paradox.

5. Prediction Equation & Residuals

  • Prediction:
    ŷ = a + b₁x₁ + b₂x₂ + … + bₖxₖ

  • Residual: y – ŷ

  • SSE: Σ(y – ŷ)²

  • Least squares chooses coefficients that minimize SSE.

6. Case Study: Mental Health in Alachua County

  • Predictors:

    • x₁ = Life events

    • x₂ = SES

  • Regression equation:
    ŷ = 28.230 + 0.103x₁ – 0.097x₂

  • Example: For x₁ = 46, x₂ = 84 → ŷ = 24.8; residual = –7.8.

7. R and R²

  • R: Correlation between observed y and predicted ŷ.

  • R²: Proportion of variance in y explained by all predictors.

  • Formula:
    R² = (TSS – SSE) / TSS

  • Properties:

    • Between 0 and 1

    • Cannot decrease when adding predictors

8. Multicollinearity

  • Predictors highly correlated with each other → unstable coefficients.

  • Ideal: predictors strongly related to y but weakly related to each other.

  • Rule of thumb: sample size ≥ 10 × number of predictors.

9. Inference for Regression Coefficients

  • Global F‑test: tests all βᵢ = 0.

  • Individual t‑tests: test βᵢ = 0.

  • df = n – (k + 1)

10. ANOVA Components

  • Conditional SD:
    s = √(SSE / [n – (k + 1)])

  • MSE = Residual Mean Square

  • F = Regression MS / Residual MS

  • Relationship: F = t² when df₁ = 1.

11. Interaction

  • Interaction model:
    E(y) = α + β₁x₁ + β₂x₂ + β₃x₁x₂

  • Slope of x₁ depends on x₂:
    β₁ + β₃x₂

  • Centering predictors improves interpretability.

12. Comparing Models

  • Complete model vs. reduced model

  • F‑test for comparing SSE values:
    F = [(SSEᵣ – SSE꜀)/df₁] / [SSE꜀/df₂]

13. Partial Correlation

  • Measures association between y and x₁ controlling for x₂.

  • Formula:
    rᵧₓ₁·ₓ₂ = (rᵧₓ₁ – rᵧₓ₂ rₓ₁ₓ₂) / √[(1 – rᵧₓ₂²)(1 – rₓ₁ₓ₂²)]

14. Standardized Coefficients (Beta Weights)

  • β* = bᵢ × (sₓᵢ / sᵧ)

  • Allows comparison across predictors with different units.

  • Standardized prediction equation has no intercept.

What is the purpose of a multivariable regression model?
To improve prediction of y and analyze relationships while controlling for other variables.

What is the general form of a multiple regression function with two predictors?
E(y) = α + β₁x₁ + β₂x₂.

What does α represent in a regression model?
The population mean of y when all predictors equal 0.

What does a β coefficient represent?
The change in mean of y for a one‑unit increase in a predictor, holding all others constant.

What is Simpson’s Paradox in the education–crime example?
The effect of education reverses direction when controlling for urbanization.

What is a residual?
The difference between observed y and predicted ŷ.

What is SSE?
The sum of squared residuals: Σ(y – ŷ)².

What is the prediction equation for the mental health study?
ŷ = 28.230 + 0.103x₁ – 0.097x₂.

What does R measure?
The correlation between observed y and predicted ŷ.

What does R² measure?
The proportion of variance in y explained by all predictors.

Can R² decrease when adding predictors?
No, R² cannot decrease when a predictor is added.

What is multicollinearity?
A condition where predictors are highly correlated with each other.

What is the global F-test used for?
Testing whether all β coefficients equal 0.

What is the t-test used for in regression?
Testing whether an individual β coefficient equals 0.

What is the formula for conditional standard deviation s?
s = √(SSE / [n – (k + 1)]).

What is the relationship between F and t for a single predictor?
F = t².

What is an interaction term?
A term like β₃x₁x₂ indicating that the effect of one predictor depends on another.

Why are variables centered in interaction models?
To make coefficients interpretable at the mean of other predictors.

What is a complete model?
A model containing all predictors.

What is a reduced model?
A simpler model nested within the complete model.

What is partial correlation?
The association between y and one predictor while controlling for others.

What is a standardized regression coefficient?
The effect of a one‑SD increase in a predictor on y, in SD units.

Why does the standardized regression equation have no intercept?
Because standardized variables have mean 0, so predicted z is 0 when all predictors are 0.