Multiple Regression Lecture Notes Flashcards

A set of vocabulary flashcards covering key terms, statistical criteria, and formulas for standard, hierarchical, and stepwise multiple regression based on the lecture notes.

Criterion variable

The outcome or dependent variable (denoted as Y) that a researcher is trying to predict.

Predictor variable

The independent variable (denoted as X) used to predict the score of the criterion variable.

Bivariate regression

A regression model that involves only one predictor variable.

Multiple regression

A statistical technique that allows for the prediction of a criterion variable from scores on two or more predictor variables.

Standard multiple regression

Also known as simultaneous regression or the 'Enter' method, this is where all predictor variables are entered into the model at the same time on equal footing.

Tabachnick & Fidell (2007) formula

The formula used to determine required sample size for regression: $N > 50 + 8M$, where $M$ is the number of predictor variables.

Multicollinearity

A problematic state where predictor variables are highly inter-correlated ($r > \pm.90$), making it difficult to determine the unique contribution of each predictor.

Singularity

A condition where there is a perfect linear relationship between predictors, which must be avoided entirely in regression models.

Tolerance

A collinearity diagnostic in SPSS that should be higher than $0.50$; values below this indicate multicollinearity concerns.

Variance Inflation Factor (VIF)

A collinearity statistic where values should be less than $10$; values above this indicate multicollinearity is a concern.

Residual

The difference between the value predicted by the regression model and the actual observed value.

Homoscedasticity

The assumption that the residuals at each level of the predictor have similar or equal variances.

Heteroscedasticity

A violation of assumptions where the spread of residuals is inconsistent across predicted values, often appearing as a fan, funnel, or bow-tie shape in a scatterplot.

Standardised Residuals

A statistic used to identify outliers; according to Tabachnick & Fidell, these should fall within the range of $\pm3.3$.

Cook's Distance

A measure of a specific case's influence on the regression coefficients; the maximum value should be less than $1$.

Adjusted $R^2$

A measure of the proportion of variance explained by the model that adjusts for the number of predictors and sample size; it is the value that should always be reported for variance explained.

Standardised Beta ($\beta$)

Regression coefficients that are converted to the same scale, allowing for direct comparison of the relative strength of different predictors.

Unstandardised B

The coefficient used to write the regression equation, representing how many units the criterion changes per unit increase in the predictor.

Regression Equation

The formula used to predict the criterion: $\hat{Y} = b_0 + b_1x_1 + b_2x_2 ... + \text{error}$, where $b_0$ is the intercept and $b_1, b_2$ are unstandardised coefficients.

Dummy variable

A categorical predictor that has been dichotomously coded (e.g., $1$ for 'Yes' and $0$ for 'No') for inclusion in a regression model.

Hierarchical multiple regression

Also called sequential regression, this method involves entering predictors into the model in blocks based on a specific order determined by the researcher's theory or prior evidence.

$R^2$ Change

The amount of additional variance in the criterion variable explained by adding a new block of predictors in hierarchical or stepwise regression.

Stepwise multiple regression

An exploratory method where the computer algorithm selects which predictors to include and their order of entry based purely on statistical criteria rather than theory.

Overfitting

A risk in stepwise regression where a model fits the current sample very well but may generalise poorly to new, independent data sets.

Data transformation

A technique used to correct distributional problems like skewness or unequal variances by changing all scores on a variable (e.g., using Square root or Log transformations).