Week 9: Correlations and Introduction to Regression

Pearson Correlation (r)

Measures the strength and direction of a linear relationship between two numeric variables.

Spearman's Rho (ρ)

A non-parametric correlation similar to Pearson's but does not assume a linear relationship; measures the strength and direction of a monotonic relationship.

Linear Regression

A tool for describing relationships between multiple interval scale (numeric) variables, with one outcome and one or more predictors.

Regression Line

The 'best fit' line through data points.

Regression Model

Acknowledges random variation in data in addition to the regression line.

Principle of 'Least Squares'

The 'best fit' regression line is the one that minimizes the sum of squared deviations between predictions and actual values; this is also called the residual sum of squares.

lm() function

Estimates the values of the intercept (b0) and slopes (b1, etc.) in R.

summary() function

Runs hypothesis tests on the lm object in R.

Multiple Regression

Involves two or more predictor variables.

Interaction Terms

Describe how the relationship between an outcome and one predictor changes depending on the level of another predictor.

Positive Interaction

Y is high when predictors have the same sign (both high or both low).

Negative Interaction

Y is low when predictors have the same sign (both high or both low).

Formula syntax for interaction

outcome ~ predictor1 * predictor2 or outcome ~ predictor1 + predictor2 + predictor1:predictor2.

Null Hypothesis in Regression

No regression line; no relationship between predictors and outcome.

Model Sum of Squares (SSm)

Difference between regression line predictions and mean Y.

Residual Sum of Squares (SSr)

Difference between data and regression line predictions.

Testing Specific Predictor Significance

Uses t-tests for each predictor to test the null hypothesis that a specific predictor has no relationship to the outcome.

Effect Size Measure

How well the overall regression model accounts for the outcome variable, depending on variance accounted for by the model vs. total variance.

Standardized Regression Coefficients

Used when scales are not standardized; standardCoefs() function (from lsr package).