Regression Fundamentals Flashcards

These flashcards cover the fundamental concepts of regression analysis, including the CEF, OLS estimation properties, FWL theorem, bias, and inference based on the BUSN 5000E lecture notes.

National Longitudinal Survey of Young Men (NLSYM)

A longitudinal study that began with $5525$ men between $14$-$24$ years old in $1966$, which served as the data source for David Card's study.

Estimation

One of the two related activities in frequentist learning used to determine the specific numerical effect of variables, such as education on wages.

Inference

The process of drawing conclusions about a population, such as the rate of return to education, using data from a random sample.

Random Sample

A collection of independent and identically distributed ($iid$) random variables that take on particular values upon observation.

Conditional Expectation Function (CEF)

The expectation of a dependent variable $y$ given an independent variable $x$, denoted as $E(y|x)$ or $\mu(x)$, used to summarize the relationship between the two.

CEF Decomposition Property

The principle that any random variable $y$ can be split into two parts: the part explained by $x$ (the CEF) and a part uncorrelated with any function of $x$ ($u$), such that $y = E(y|x) + u$.

Unconditional Expectation

The population mean of a variable, denoted by $\mu$ or $E(y)$, which is a constant and not a function of another variable.

Population Regression Function (PRF)

The linear approximation to the CEF, written as $E(y|x_1, \dots, x_K) = \beta_0 + \beta_1 x_1 + \dots + \beta_K x_K$.

Population Regression Coefficients

The parameters $\beta_0$ and $\beta_1$ that solve the population least-squares problem by minimizing the mean squared error (MSE).

Best Linear Predictor

One of the three justifications for regression, stating that the PRF provides the most accurate linear prediction of $y$ given the $x$ variables.

Ordinary Least-Squares (OLS) Estimators

The sample analogs to population regression coefficients, denoted as $\hat{\beta}_0$ and $\hat{\beta}_1$, calculated by replacing population expectations with sample averages.

Consistency

An estimator property where the estimate converges to the underlying population estimand as the sample size grows toward infinity.

Law of Large Numbers (LLN)

Succinctly described as the principle that sample averages converge toward their population counterparts for increasingly large samples.

Rate of Return

The interpretation of the coefficient $\beta_1$ when the dependent variable is measured in logs ($\log$ wages) and the regressor is years of schooling.

Residualized Variable

That part of an explanatory variable ($\hat{r}_{i1}$) remaining after accounting for the influence of all other explanatory variables in the model.

Frisch-Waugh-Lovell (FWL) Theorem

The theorem stating that the coefficient of a residualized $x_1$ in a simple regression is identical to the coefficient of $x_1$ in a multiple regression including all controls.

Partial Effect

The change in the dependent variable associated with a unit change in one explanatory variable while holding all other factors and the error term constant.

Total Sum of Squares (SST)

The total variation in the dependent variable, decomposed as $SST = SSR + SSE$.

R-squared ($R^2$)

A goodness-of-fit measure defined as $SSR/SST$, representing the proportion of variation in $y$ accounted for by its fitted values.

Unbiasedness

A property requiring that the expected value of the estimator equals the true population estimand, meaning sampling error is zero on average.

Conditional Mean Assumption (CMA)

The critical condition for unbiasedness stating that regression errors are mean independent of the explanatory variables, or $E(u_i|x_{i1}, \dots, x_{iK}) = 0$.

Omitted Variable Bias (OVB)

The bias introduced to coefficient estimates when a confounding variable that is correlated with the included regressors is excluded from the model.

OVB Formula

The expression $E(\tilde{\beta}_1) = \beta_1 + \beta_2 \tilde{\delta}$, where the bias depends on the effect of the omitted variable and its correlation with the included regressor.

Classical Measurement Error

Error in an explanatory variable that typically biases the estimated regression coefficient toward zero, known as downward bias.

Sampling Distribution

The distribution of an estimator, such as $\hat{\beta}_k$, that would be obtained through infinite repeated sampling from the population.

Central Limit Theorem (CLT)

The theorem used to approximate the sampling distribution of the OLS estimator as asymptotically normal with mean $\beta_k$ and variance $var(\hat{\beta}_k)$, provided the estimator is consistent.

Homoscedasticity

The classical assumption that the variance of the regression error $u$ is constant and does not depend on the values of the explanatory variables.

Heteroscedasticity-Robust Standard Errors

Standard errors that provide valid inference even if the error variance depends on the explanatory variables, easily computed using the vcov option.

Significance Level ($\alpha$)

The probability of committing a Type I error, commonly set at $0.05$, which represents the risk of falsely rejecting a true null hypothesis.

Type II Error

A false negative result, occurring when there is a failure to reject a null hypothesis that is actually false.

Test of Significance

A common hypothesis test where the null hypothesis is that a coefficient is zero ($H_0 : \beta_k = 0$).

p-value

The marginal significance level, or the probability of drawing a test statistic at least as extreme as the one calculated, assuming the null hypothesis is true.

F-test

A statistical test used to evaluate the joint significance of multiple estimated regression coefficients by comparing the fits of long and short models.

Average Partial Effect (APE)

The partial effect of an independent variable evaluated at its sample mean, particularly useful in models with quadratics or interactions.

Linear Probability Model (LPM)

A regression model where the dependent variable is binary, representing the response probability $P(y = 1|x)$, though it is heteroscedastic by construction.