CHAPTER 3 Causal Inference and the Four Hurdles of Causality in Political Science

Foundation of Causal Inference in Political Science

• Correlation versus Causation:

  • A statistical correlation indicates an association between two variables, but it does not inherently establish a causal connection.

  • ESP Survey Example: Survey data showing that certain individuals identify stimuli unperceived by others may demonstrate statistical correlations, but without experimental evidence, these remain mere correlations rather than causal relationships.

  • Spurious Correlation Example: A strong statistical correlation exists between declining deaths from falling out of fishing boats over time and the declining marriage rate in Kentucky. Despite matching longitudinal trends, there is zero causal connection between these two phenomena.

  • Theory building requires a sensible, logical mechanism; empirical correlation alone cannot justify a causal hypothesis.

• Daily Language versus Social Science Complexity:

  • Daily language and political rhetoric favor binary, simplistic answers (e.g., binary choices represented quantitatively as $0$ or $1$, such as binary stances on gun control policy, trade wars, or candidate support).

  • Political science studies human society and complex political behavior, where a dependent variable ($Y$) is driven by multiple factors ($X_1, X_2, X_3, X_4, X_5$ or $X, Z, W, \beta, \text{etc.}$).

• Theoretical Definition:

  • A political science theory defines a formal relationship between an independent variable ($X$) and a dependent variable ($Y$), establishing how $X$ causes or affects $Y$.

  • Because political outcomes ($Y$) are multicausal, isolated theories focusing on a single independent variable ($X$) provide limited explanatory power, making rigorous research design essential.

Deterministic versus Probabilistic Causal Relationships

• Deterministic Relationships in Physical Sciences:

  • Originating in physical sciences such as physics, deterministic relationships dictate that using identical materials, environments, and experimental procedures will consistently produce identical, replicable outcomes.

• Probabilistic Relationships in Social Sciences:

  • Human beings are non-deterministic, non-robotic actors whose behaviors do not conform to law-like deterministic statements.

  • Identical experimental procedures, instructions, and decision tasks across different social science laboratories yield non-identical outcomes and vastly different choices.

  • In experimental psychology and sociology, replication is a major challenge: up to $90\%$ of groundbreaking experimental psychological findings fail to replicate across laboratories.

  • Causal assertions in social and political sciences are strictly probabilistic rather than deterministic.

• Examples of Probabilistic Relationships:

  • Income and Tax Rate Preferences:

  • Empirical data demonstrates a probabilistic trend where higher individual income increases the likelihood of preferring lower income tax rates to protect personal wealth.

  • This relationship is non-deterministic: wealthy individuals may advocate for higher tax rates due to charitable motivations, tax avoidance knowledge, or ideological beliefs.

  • Democratic Peace Theory:

  • Data shows that two democratic nations are significantly less likely to go to war with one another than pairs of states where at least one nation is non-democratic.

  • This is probabilistic: democratic nations have historically engaged in military conflicts, but the probability of bilateral conflict between democracies is markedly lower.

  • Mechanism (Audience Costs): Leaders in democratic regimes face high audience costs because losing an international conflict severely harms their electoral standing and reelection prospects. Because war is a zero-sum or lose-lose endeavor, two democratic leaders subject to audience costs exercise extreme caution. In contrast, authoritarian leaders face minimal electoral audience costs and may instigate international conflict to temporarily boost domestic approval ratings during international crises.

• Necessity of Statistical Methods:

  • Because political science causal claims are probabilistic, statistical analysis is required to evaluate probabilities, likelihoods, and comparative empirical trends.

The Four Hurdles of Establishing Causality

• Overview:

  • The textbook ($3\text{rd}$ edition) establishes a mandatory four-hurdle framework for evaluating whether an independent variable ($X$) causally impacts a dependent variable ($Y$).

  • Generative AI tools often provide incorrect or generalized alternatives to these specific four hurdles; research design and unit evaluations require strict adherence to this textbook formulation.

• The Four-Hurdle Framework:

  1. Hurdle 1 (Theoretical): Is there a credible causal mechanism connecting $X$ to $Y$?

  2. Hurdle 2 (Theoretical): Can we rule out the possibility that $Y$ causes $X$ (reverse causality)?

  3. Hurdle 3 (Empirical): Is there covariation between $X$ and $Y$?

  4. Hurdle 4 (Empirical): Have all possible confounding variables ($Z$) been controlled for?

• Role of Operationalization:

  • Theoretical analysis encompasses Hurdles 1 and 2, while empirical analysis encompasses Hurdles 3 and 4. Operationalization serves as the bridge connecting theoretical concepts to empirical metrics.

• Detailed Breakdown of Each Hurdle:

  • Hurdle 1: Credible Causal Mechanism:

  • Requires a plausible theoretical explanation detailing how and why $X$ drives $Y$.

  • Failing Hurdle 1 requires either discarding the hypothesis entirely or fundamentally rethinking the underlying causal pathway.

  • Must answer "Yes" to Hurdle 1 to proceed with research design.

  • Example: Economic performance ($X$) affects employment and individual income, which subsequently alters individual support for the incumbent administration ($Y$). Identifying employment/income as the intermediary mechanism satisfies Hurdle 1.

  • Hurdle 2: Ruling Out Reverse Causality ($Y \rightarrow X$):

  • Must verify that the dependent variable $Y$ does not cause the independent variable $X$.

  • Reciprocal Causality ($X \rightleftharpoons Y$): Common in political science, where $X$ causes $Y$ and $Y$ simultaneously influences $X$ (e.g., strong economic performance increases incumbent support, and high incumbent support enables aggressive economic policies that further alter performance).

  • Resolving Reciprocal Causality: Researchers isolate directionality by analyzing data within a specific, fixed time window, as policy changes and economic outcomes take time to unfold.

  • Cell Phone and Cancer Example: Longitudinal data shows rising cell phone usage correlating with total cancer incidence. However, examining a specific time window where cell phone usage spiked exponentially reveals a flat cancer incidence rate, ruling out direct causation during that window.

  • Gender and Voting Example: Biological gender ($X$) influences voting behavior ($Y$). Reverse causality ($Y \rightarrow X$) is impossible because voting choices cannot alter biological gender.

  • Hurdle 3: Covariation between $X$ and $Y$:

  • Requires empirical demonstration that changes in $X$ systematically correspond to changes in $Y$ ($X$ and $Y$ co-vary).

  • While correlation does not equal causation, covariation is an absolute prerequisite for causation. A flat, horizontal regression line showing no change in $Y$ as $X$ varies indicates a lack of covariation, failing Hurdle 3.

  • Valid causal inference strictly requires an affirmative "Yes" to both Hurdle 1 (theoretical requirement) and Hurdle 3 (empirical requirement).

  • Hurdle 4: Controlling for Confounding Variables ($Z$):

  • A confounding variable ($Z$) is an alternative factor that influences both $X$ and $Y$, or influences $Y$ independently, creating a false appearance of direct causality between $X$ and $Y$.

  • Ice Cream and Murder Example: Ice cream consumption ($X$) correlates with murder rates ($Y$). Outdoor temperature ($Z$) is the true confounding variable: warmer weather increases ice cream sales and brings more people outdoors, elevating murder rates.

  • Judicial Politics Example: Studies show that judges with daughters ($Z$) issue demonstrably more lenient court rulings ($Y$) compared to judges without daughters.

  • Stock Market Example: Oil prices ($X$) correlate with stock market performance ($Y$), but presidential social media posts ($Z$) responding to oil prices directly drive stock market movements.

  • Comparative Statics / Ceteris Paribus ("All else equal"): Researchers cannot observe or measure every latent variable in existence. Research designs strive for ceteris paribus (comparing "apples to apples") by holding potential confounding variables constant to isolate the impact of varying $X$ on $Y$.

Evaluation of Causal Claims: Case Studies

• Case 1: Private Schooling versus Standardized Test Performance:

  • Variables: Independent variable $X$ = Attending private vs. public school (school choice participation); Dependent variable $Y$ = Standardized test scores (SAT, PSAT, AP, IB).

  • Hurdle 1 (Mechanism): Yes — Private schools often feature smaller class sizes, allowing greater individual instruction that improves test performance.

  • Hurdle 2 (Reverse Causality): Yes — Standardized tests take place months or years after the choice to attend a private school, preventing future test scores from causing past enrollment choices.

  • Hurdle 3 (Covariation): Yes — Empirical data confirms that private school students achieve higher average standardized test scores.

  • Hurdle 4 (Confounding Variables): Failed / No — Confounded by parental income, education, and involvement ($Z$) (selection bias). Wealthy, highly involved parents are more likely to afford private school tuition, and parental involvement directly drives student academic achievement regardless of school type.

• Case 2: Mass Public Life Satisfaction and Democratic Stability:

  • Variables: $X$ = Mass public life satisfaction; $Y$ = Stability of democratic institutions.

  • Hurdle 1: Yes — Highly satisfied citizens are less motivated to overthrow government institutions.

  • Hurdle 2: Failed / No — Citizens living in stable democracies naturally report higher life satisfaction ($Y \rightarrow X$), generating reciprocal causality.

  • Hurdle 3: Yes — Empirical covariation is present.

  • Hurdle 4: Failed / No — Confounded by overall economic development, historical institutions, culture, education, and social capital ($Z$).

• Case 3: Race and Political Participation in the United States:

  • Variables: $X$ = Race (Anglo, African American, Latino); $Y$ = Political participation (number of participatory acts during an election cycle).

  • Empirical Observation: Typical election cycles reveal Anglos engage in $2.2$ participatory acts, African Americans engage in $1.9$ acts, and Latino citizens engage in $1.41$ acts.

  • Hurdle 1: Yes — Historical, formal, and informal institutional barriers discourage political participation among non-Anglo populations.

  • Hurdle 2: Yes — Political participation cannot alter biological race.

  • Hurdle 3: Yes — Empirical data confirms participation variance across racial groups.

  • Hurdle 4: Failed / No — Confounded by socioeconomic status (SES) ($Z$). Race is heavily correlated with SES in the United States. When controlling for SES, the observed statistical difference in political participation rates across racial groups disappears completely.

• Case 4: Head Start Program and Kindergarten Performance:

  • Variables: $X$ = Participation in Head Start (preschool program for underprivileged children); $Y$ = Academic success in kindergarten and primary school.

  • Hurdle 1: Yes — Early exposure to structured learning environments enhances skill development.

  • Hurdle 2: Yes — Primary school performance cannot cause past preschool enrollment decisions.

  • Hurdle 3: Yes — Empirical data confirms higher academic success among participants.

  • Hurdle 4: Failed / No — Confounded by parental involvement ($Z$). Parents who proactively enroll children in Head Start may inherently provide higher developmental support at home.

Questions & Discussion

• Reverse Causality versus Confounding Selection in Private Schooling:

  • Question: Could pre-existing student academic performance or scholarship offers from private schools mean that student test capability causes private school attendance ($Y \rightarrow X$)?

  • Explanation: Reverse causality specifically requires the future outcome $Y$ (e.g., SAT scores) to directly cause the prior action $X$ (school choice). Because standardized tests occur long after enrollment, future scores cannot cause past enrollment. However, pre-existing student talent and scholarship offers act as confounding $Z$ variables (selection bias) operating prior to enrollment, which must be controlled for under Hurdle 4.

• Experimental Control Groups for School Choice Studies:

  • Question: How can a research design control for scholarship selection bias ($Z$)? Would creating control groups of non-scholarship students versus treatment groups of scholarship students resolve Hurdle 4?

  • Explanation: Isolating students into scholarship and non-scholarship groups across both public and private institutions effectively controls for scholarship status as a $Z$ variable.

  • Follow-up: Is it necessary to sample average public school students as well?

  • Explanation: Yes, a comprehensive comparison requires data from students across public and private schools, while controlling for additional variables such as age, class size, racial composition, and parental volunteer hours.