Measurement Error and Sample Selection

Flashcards covering measurement error (classical vs. classification), sample selection issues (MCAR, MAR, endogenous selection), and terminology from the Project STAR study notes.

Project STAR

Short for the Tennessee Student/Teacher Achievement Ratio experiment, it was the largest randomized experiment of its kind beginning in the 1985-86 school year, randomly assigning kindergarteners and teachers to small, regular, and regular+aide class configurations.

Education Production Function

A conceptual model for identifying the relationship between educational inputs (like class size) and outcomes (achievement), which Krueger (1999) notes is often uncertain because of specification issues and omitted variables.

Consistency

A property of an estimator where bias and sampling error vanish and the estimator collapses onto the underlying estimand as the sample size grows.

Measurement Error

Inaccurate data measurements of the phenomenon of interest, defined by the relationship $Measurement = Truth + Error$, or $X = X^* + e$.

Classical Measurement Error Assumptions

The assumptions that 1) $E(e) = 0$, 2) the truth and error are independent ($f(e|X^*) = f(e)$), and 3) $cov(Y, e) = 0$.

Classification Error

A non-classical form of measurement error that occurs when categorical data (specifically dummy variables like $D^*$) is misclassified, meaning the error $e$ must be negatively correlated with the truth.

False Positive (Classification)

The probability that a unit is recorded as having a characteristic or being in a treatment group when it is not, denoted as $P(D = 1|D^* = 0)$.

False Negative (Classification)

The probability that a unit is recorded as not having a characteristic or not being in a treatment group when it actually does, denoted as $P(D = 0|D^* = 1)$.

Law of Total Probability (LTP)

A principle used to decompose the probability of an observed assignment (like being assigned to a small class) into the sum of conditional probabilities involving the truth and error rates: $P(D = 1) = P(D = 1|D^* = 1)P(D^* = 1) + P(D = 1|D^* = 0)P(D^* = 0)$.

Selected Sample

A sample that is not randomly drawn from the population, arising from mechanisms like sample design, nonresponse, or attrition.

Missing Completely at Random (MCAR)

A condition where the selection indicator $S$ is independent of both $Y$ and $X$, meaning $P(S = 1|Y, X) = P(S = 1)$.

Missing at Random (MAR)

A case where the selection process can depend on $X$ but not other unobserved factors, defined as $E(Y|X, S) = E(Y|X)$; this is also known as exogenous selection.

Endogenous Selection

A source of bias where sample selection is based on the outcome variable $Y$, making it problematic to learn about the Conditional Expectation Function $E(Y|X)$.

Topcoding

A specific type of censoring problem where high-value observations (such as earnings) are surveyed but recorded at a fixed threshold value.

Item Non-response

A situation where a unit is included in the sample but has missing values for specific variables.

Unit Non-response

A situation where an entire unit from the population is omitted from the sample.

Imputation

The process of filling in missing values based on observed data using relational, longitudinal, or statistical models.

Allocation Flags

Indicators used in data files like the CPS to identify whether a value was originally reported by the respondent or was subsequently imputed.

Hot Deck

A statistical technique used for imputing missing values, specifically mentioned as a method used for allocation in the CPS.

aov

The analysis of variance function in R used to compute the F statistic for testing multiple equality restrictions simultaneously (e.g., $μ_1 = μ_2 = μ_3$).

lm

The linear model function in R used for regression analysis, which can also provide F-statistics and t-tests for separate class-size effects.