Causal Inference, Potential Outcomes Framework, and Quantifying Uncertainty

Vocabulary practice flashcards covering research design, the potential outcomes framework, experimental methods, statistical noise and bias, standard errors, t-statistics, p-values, and confidence intervals.

Hypothesis

A testable prediction that should or should not hold if a theory is true, serving to simplify reality and identify trends.

Treatment (Independent Variable)

An intervention whose effects are the object of study; the factor that researchers hypothesize does the causing.

Outcome (Dependent Variable)

The variable that researchers hypothesize the treatment has an effect on.

Treatment Effect

The causal effect of the treatment on the outcome, calculated as the difference between the outcome observed under treatment and the outcome observed under control.

Potential Outcomes

Completely hypothetical constructs describing what would have happened to a particular individual or unit if they received versus did not receive a specific treatment.

Causality

The difference between potential outcomes for the exact same unit under alternative treatment states.

Counterfactual

What would have occurred to a unit had a specific condition, action, or treatment not happened.

Observational Data

Data in which researchers do not know what determined whether specific individuals received the treatment and others did not.

Experimental Data

Data in which whether individuals received a treatment was determined entirely by randomization.

Omitted Variable Bias

A systematic distortion occurring when a third factor causes both $A$ and $B$, making them correlated without a direct causal link; synonymous with selection bias where participants differ systematically.

Democratic Peace Theory

An international relations theory based on the correlation that pairs of democracies are less likely to go to war with each other than other pairs of states (such as autocracy-autocracy or democracy-autocracy).

Reverse Causation

The possibility that an observed correlation between two variables occurs because the presumed outcome ($B$) actually causes the explanatory variable ($A$).

Covariate Balance

The condition achieved through random assignment where treatment and control groups show no systematic differences across observed and unobserved baseline characteristics.

Calculus of Voting Model

A decision-theoretic model of turnout formulated as $(P \times B) + D > C$, where $P$ is the probability of deciding the election, $B$ is the benefit of a preferred candidate winning, $D$ is the benefit from fulfilling civic duty, and $C$ is the cost of voting.

Hawthorne Condition

An experimental treatment arm in the 2006 Michigan primary voter study that exerted social pressure by informing recipients: "You're being studied".

Neighbors Condition

An experimental treatment arm in the 2006 Michigan primary study that applied social pressure by mailing households their own past voting turnout record alongside the records of all their neighbors.

True Average Treatment Effect (ATE)

The exact causal effect of a treatment across a population that would be observed if every unit's potential outcomes under both treatment and control were known.

Estimate

The empirical value obtained from a single run of a specific study, serving as the best guess of the true Average Treatment Effect.

Bias

A systematic deviation from the truth in a consistent direction; a flawed study design that yields inaccurate results regardless of how large the sample size becomes.

Noise

Random variation causing sample estimates to bounce around the truth purely due to chance, which diminishes toward zero as sample size approaches infinity.

Standard Deviation

A statistical metric of spread quantifying how far individual observations in a dataset deviate on average from their sample mean.

Standard Error

A metric measuring how far an experimental estimate is expected to differ from the truth across repeated hypothetical studies due to random chance.

Sample Size Square Root Rule

The mathematical principle dictating that the standard error scales inversely with the square root of sample size; reducing the standard error by a factor of $\frac{1}{x}$ requires expanding the sample size by a factor of $x^2$.

t-statistic

A standardized metric calculated as $t = \frac{\text{estimate}}{\text{standard error}}$, used to evaluate how extreme an observed estimate is relative to random noise under the assumption of no effect.

Null Hypothesis ($H_0$)

The default hypothesis posited by a skeptic that researchers seek to disprove, typically asserting that the treatment has zero causal effect ($\text{true ATE} = 0$).

Alternative Hypothesis ($H_a$)

The hypothesis researchers aim to demonstrate, asserting that the treatment does have a causal effect ($\text{true ATE} \neq 0$).

p-value

The probability of obtaining an estimate as large or larger (as extreme or more extreme) than the observed value, assuming the null hypothesis were true.

Statistical Significance

An agreed-upon convention indicating strong statistical evidence against the null hypothesis, achieved when a $p$-value is less than $0.05$ (or a $t$-statistic exceeds $\pm 1.96$).

95% Confidence Interval

A range defined by $\text{estimate} \pm (1.96 \times \text{SE})$

In 95% of experiments, the confidence interval will contain the true treatment effect

If we repeated the experiment many times, 95% of the estimated confidence intervals would contain the true ATE

Heterogeneous Treatment Effect

Differences in the causal effect of a treatment across distinct subgroups of observations within a study population.

P-hacking

The problematic practice of repeatedly testing multiple subgroups or model specifications until finding an artificial statistically significant result ($p < 0.05$) by chance.