The Beast of Bias in Regression Analysis

A collection of flashcards focused on key concepts related to bias in regression analysis, including outliers, influential points, and model assumptions.

Outliers

Data points that lie far from the rest, which may or may not affect the integrity of a model.

Influential Points

Outliers that significantly change the regression line if removed, indicating they strongly affect the model's estimates.

Standardized Residuals

Residuals scaled so that typical values fall within ±2 or ±3; values greater than 3 may be considered outliers.

DF Beta

A measure of how much the regression coefficient would change if a particular case were removed; a value greater than 1 suggests high influence.

Cook’s Distance

A metric measuring the overall influence of a data point on the fitted values; values greater than 1 indicate a potential red flag.

Linearity

The assumption that the true relationship between predictors and the outcome is linear.

Homoscedasticity

The assumption that the variance of errors is constant across all levels of the predictor.

Spherical Errors

Assumption that the errors are independent and identically distributed with constant variance.

Normality of Errors

The requirement that residuals, not the data itself, should follow a normal distribution to validate inferential tests.

Robust Regression

A regression method that is less sensitive to outliers, allowing for more reliable estimates.

Bootstrap

A resampling technique that involves repeatedly drawing samples from a dataset and calculating estimates to form confidence intervals.

Heteroscedasticity-consistent Standard Errors

Adjusted standard errors used in regression analysis to account for non-constant variance in the errors.