Chapter 11: Multiple Regression and Correlation Flashcards

A set of practice flashcards defining key vocabulary and concepts for multiple regression, correlation, and model inference from Chapter 11.

Bivariate Model

A regression model containing a single explanatory variable and a single response variable, represented by the equation $E(y) = \alpha + \beta x$.

Multiple Regression Model

A model that describes the relationship between a response variable $y$ and a collection of explanatory variables ($x_1, x_2, \dots, x_k$).

Partial Regression Equation

An equation obtained by fixing all but one explanatory variable at particular levels to relate the mean of $y$ to the remaining predictor variable.

Simpson's Paradox

A phenomenon where the direction of a partial association between variables is opposite to the direction of the bivariate association between those same variables.

Partial Regression Coefficients

The parameters ($\beta_1, \beta_2, \dots, \beta_k$) in a multiple regression model that describe the effect of an explanatory variable while controlling for the other variables in the model.

Least Squares Criterion

A method for finding a prediction equation that minimizes the sum of squared errors ($SSE$) between observed and predicted values.

SSE (Sum of Squared Errors)

Also called the residual sum of squares, it is calculated as $SSE = \sum (y - \hat{y})^2$ and summarizes the closeness of fit of the prediction equation to the data.

Scatterplot Matrix

A diagram that presents scatterplots for each pair of variables in a single display, where each pair is shown twice (once with each variable on the $y$-axis).

Partial Regression Plot

A diagnostic plot that displays the relationship between a response variable and an explanatory variable after removing the linear effects of other predictors in the model.

Multiple Correlation (R)

The correlation between the observed values of $y$ and the predicted values $\hat{y}$, ranging from $0$ to $1$.

R-squared ($R^2$)

The coefficient of multiple determination, defined as $\frac{TSS - SSE}{TSS}$. It measures the proportion of the total variation in $y$ explained by all explanatory variables combined.

Total Sum of Squares (TSS)

The total variation of the $y$-values about their sample mean, calculated as $TSS = \sum (y - \bar{y})^2$.

Multicollinearity

A condition where explanatory variables in a regression model are highly correlated with one another, making it difficult to assess the unique contribution of each variable.

Global Test of Independence

A significance test checking the null hypothesis $H_0: \beta_1 = \beta_2 = \dots = \beta_k = 0$ to determine if any of the explanatory variables are related to the response variable.

F Distribution

A right-skewed sampling distribution used for global tests in regression, determined by degrees of freedom $df_1 = k$ and $df_2 = n - (k + 1)$.

MSE (Mean Square Error)

An estimate of the conditional variance of $y$ at fixed predictor values, calculated as $MSE = \frac{SSE}{n - (k + 1)}$.

Interaction

A condition where the effect of one explanatory variable on the response variable changes as the level of another explanatory variable changes.

Cross-Product Terms

Artificial variables (e.g., $x_1x_2$) added to a multiple regression model to permit the modeling of statistical interaction between predictors.

Complete Model

In model comparison, the full regression model containing all predictors (including extra terms like interactions).

Reduced Model

A simpler version of a model nested within the complete model, containing only a subset of the predictors.

Partial Correlation

A measure of the strength of the association between a response variable and an explanatory variable while controlling for one or more other variables.

Standardized Regression Coefficient

Also known as beta weights, these represent the change in the mean of $y$ (in standard deviation units of $y$) for a one-standard-deviation increase in a predictor, controlling for other variables.