Lecture 12 - Regression Diagnostics

Vocabulary flashcards covering the core regression diagnostic assumptions, assumption violation consequences, detection methods, multicollinearity diagnostics, and outlier assessments from Lecture 12.

Linear in the Parameters (Assumption 1)

The regression assumption requiring that the model parameters ($\beta$'s) do not have exponents or fractions; violation causes coefficient estimates to be biased.

Random Sampling (Assumption 2)

The assumption that the data consists of a random sample of $n$ observations drawn from the underlying population.

No Perfect Collinearity (Assumption 3)

The assumption that none of the independent variables are constants and that no exact linear relationship exists among any of the independent variables.

Zero Conditional Mean (Assumption 4)

The assumption that the expected value of the error term given any values of the independent variables is zero ($E[\epsilon \mid x_1, \dots, x_k] = 0$), implying no unobserved confounders.

Homoskedasticity (Assumption 5)

The assumption that the error term has constant variance for any given value of the independent variables ($\text{Var}(\epsilon \mid x_1, \dots, x_k) = \sigma^2$); violation leads to unreliable standard errors.

Gauss-Markov Assumptions

Assumptions 1 through 5 of multiple linear regression under which Ordinary Least Squares (OLS) estimators are proven to be the Best Linear Unbiased Estimators (BLUE).

Classical Linear Model Assumptions

The full set of six assumptions (Assumptions 1 through 6, adding normally distributed errors with zero mean and variance $\sigma^2$) required for reliable hypothesis testing.

Residual ($\hat{\epsilon}$)

The difference between the actual observed value of the dependent variable and the fitted value ($\hat{\epsilon} = y - \hat{y}$), commonly examined to detect assumption violations.

Endogenous Variables

Explanatory variables included in the regression model that are correlated with the error term, causing coefficient estimates to be biased.

Exogenous Variables

Explanatory variables included in the regression model that are uncorrelated with the error term.

Multicollinearity

A condition occurring when an independent variable can be strongly predicted by a linear relationship with other independent variables, inflating standard errors and destabilizing coefficient estimates.

Variance Inflation Factor (VIF)

A diagnostic metric given by $\text{VIF} = \frac{1}{1 - R_k^2}$ that assesses how much the variance of an estimated regression coefficient is inflated by multicollinearity.

VIF Rule of Thumb Threshold

A threshold where $\text{VIF} > 10$, indicating that a variable is causing a serious multicollinearity issue in the model.

Residuals vs. Fitted Values Plot

A diagnostic scatter plot used to assess linearity and homoskedasticity; curvature indicates non-linearity, while a non-constant spread indicates heteroskedasticity.

Quantile-Quantile (Q-Q) Plot

A diagnostic plot graphing studentized residuals against theoretical normal distribution quantiles to verify the normality of errors assumption.

Index Plot of Residuals

A plot of residuals ordered by an index (such as time or location) used to check the independence of errors; an observable pattern indicates autocorrelation.

Autocorrelation of Errors

A violation of the independence assumption identified by patterns across ordered observations in an index plot of residuals, common in time-series analysis.

Outlier

A data point that does not have a good fit with respect to the observed values and the estimated regression line (typically having a large residual).

Influential Observation

A data point that may have a good fit but disproportionately dictates the slope of the regression line and how the relationship is perceived.

Cook's Distance

A diagnostic metric used to detect influential observations and outliers, typically flagging problematic points when values exceed the threshold \frac{4}{\text{# of obs}}.

Potential Confounder (Variable Inclusion Situation 3)

A scenario where adding $x_2$ yields a statistically significant coefficient on $x_2$ and substantially alters the coefficient on $x_1$, meaning $x_2$ must be included in the model.

Variance-Reducing Predictor (Variable Inclusion Situation 4)

A scenario where the coefficient on a new variable $x_2$ is significant while the coefficient on $x_1$ remains stable; $x_2$ is included because it reduces error variance.