Correlation and Simple Linear Regression

Purpose of correlation and simple linear regression

To describe relations, predict values, and indicate the strength of association between two continuous variables.

Difference between correlation and regression

Correlation indicates the strength and direction of association as a single number; regression models the relationship to predict or explain the value of one variable from another.

Statistical model

A formalized mathematical equation that describes and represents the relationship between variables as a simplification of reality, using probability distributions for variability.

Pearson's correlation coefficient (r)

A standardised index ranging from -1 to +1 indicating the degree of straight-line association between two continuous variables.

What does a correlation of zero indicate?

It indicates no linear relation between the values of the two variables (though a non-linear relation may still exist).

Why is a scatterplot essential in correlation and regression?

To visually inspect the data for linear trends, patterns, and outliers before relying solely on numerical coefficients.

Regression line

A straight line passing through the data that best describes and summarises the relationship between independent (x) and dependent (y) variables.

Least square distance criterion (LSD)

The criterion used to find the best-fit regression line by minimising the unexplained variance (residuals).

Equation of a simple linear model

y = β0 + β1·x + ε, where β0 is the intercept and β1 is the slope.

Meaning of β0 (intercept)

The expected value of the dependent variable (y) when the independent variable (x) is zero.

Meaning of β1 (slope)

The expected change in the dependent variable (y) for a one-unit increase in the independent variable (x).

R-squared (R²)

The proportion of variance in the dependent variable explained by the regression model.

Null Hypothesis (H0) for regression slope

H0: β1 = 0, indicating no linear relationship between the independent and dependent variables.

Assumptions of linear regression

Linear relationship, normally distributed error terms, homogeneity of error variances (homoscedasticity), and independent errors.