Causal Inference and Policy Evaluation Lecture 1: Correlation vs Causation
Introduction to Causal Inference and Policy Evaluation
Course Metadata:
Title: Causal Inference and Policy Evaluation, Lecture 1: Correlation vs Causation.
Instructor: Vojtech Bartos, University of Milan.
Date: March–June 2026.
Motivation and Real-World Applications:
Understanding the relationship between variables is critical for evaluating claims regarding police, education, vaccines, and energy.
The core challenge is that correlation is often misleading due to underlying issues like reverse causality, omitted variables (confounding), and spurious correlations.
Key Distinctions:
Prediction: Aims to identify patterns in existing data to fit functional relationships between variables for high accuracy. The goal is low prediction error (e.g., mean squared error). It asks: "If I observe $X$, what do I expect for $Y$?"
Causation: Focuses on the effects of interventions and counterfactuals. It asks: "If I change $X$, what happens to $Y$?" Causal inference is essentially a prediction of a counterfactual associated with a particular decision.
Case Studies in Causal Claims
Renewable Energy and Electricity Costs:
Claim: Bjørn Lomborg (2026) suggests that "The more solar and wind, the costlier it gets."
Context on Lomborg: Danish political scientist, president of the Copenhagen Consensus Center, former director of the Danish government's Environmental Assessment Institute, and author of "The Skeptical Environmentalist."
Confounding Factors in Energy Pricing:
Rich Countries: Tend to have higher energy taxes (e.g., approximately in Denmark) and higher grid fees, alongside more renewables.
Developing Countries: Often provide massive subsidies for electricity (e.g., Indonesia spent annually on subsidies in 2024), which artificially lowers visible costs.
Vaccine Efficacy:
Polio: Even without certain historical trials, evidence is highly suggestive of efficacy.
Measles: Statistics from Project Tycho (2018) and the CDC (1959-2022) show a dramatic drop in reported cases per people following key milestones:
1963: John Enders develops the first measles vaccine.
1971: Maurice Hilleman develops the MMR (measles-mumps-rubella) vaccine.
1980: Mandatory vaccination for children entering kindergarten in the US.
Covid-19: Rigorous Randomized Controlled Trials (RCTs) like those by Polack et al. (2020) in the NEJM established vaccine efficiency.
Education and Wages:
Data (US Bureau of Labor Statistics, 2024):
Doctoral degree: Median weekly earnings: ; Unemployment: .
Professional degree: Median weekly earnings: ; Unemployment: .
Master's degree: Median weekly earnings: ; Unemployment: .
Bachelor's degree: Median weekly earnings: ; Unemployment: .
Associate degree: Median weekly earnings: ; Unemployment: .
Some college, no degree: Median weekly earnings: ; Unemployment: .
High school diploma: Median weekly earnings: ; Unemployment: .
Less than high school diploma: Median weekly earnings: ; Unemployment: .
Total/All Workers: Median weekly earnings: ; Unemployment: .
Causal Pathways: Education leads to higher productivity, which leads to higher wages.
Reverse Causality: Individuals with high expected wages invest more in education.
Confounders: Innate ability might cause both higher education and higher wages.
Evolutionary Thinking on Causality
David Hume (1739-1740):
Proposed a reductionist viewpoint in "A Treatise of Human Nature."
Defined a cause as an object precedent and contiguous to another, where all resembling objects are placed in like relations of precedency and contiguity. Essentially, if $Y$ generally happens before $X$, $X$ causes $Y$.
Clive Granger (1977):
Developed "Granger Causality" (Nobel Prize in Economics, 2003).
Rather than testing if $X$ causes $Y$, it tests whether lagged $X$ forecasts $Y$. It warns against the fallacy of "Post hoc ergo propter hoc" (after this, therefore because of this).
The Rooster Example: A rooster crows every morning, and the sun rises. However, the crowing does not cause the sunrise. This is demonstrated by the fact that the sun would still rise if the rooster were turned into soup.
John Stuart Mill (1843):
Offered a structural account in "A System of Logic."
Suggested that $X$ causes $Y$ if it happens before $X$ AND all other factors remain constant except the one being tested. This aligns with modern counterfactual thinking.
Jerzy Neyman (1923) and Donald Rubin (1974):
Introduced the "Potential Outcomes" framework.
Asks: "All else constant, what would have happened if the treatment was administered vs. if it was NOT administered?"
Ronald Fisher (1935):
Wrote "The Design of Experiments."
Lady Tasting Tea Experiment: A woman claimed she could tell whether milk or tea was poured first. Fisher used a design of eight cups (four of each preparation) in random order for a blind tasting.
Probability Theory: The null hypothesis () assumed her answers were due to chance. The probability of guessing all eight correctly is:
The Case of Police and Crime
Mechanisms: Potential reasons why Police (Treatment) reduces Crime (Outcome) include deterrence, faster response times, and a higher probability of arrest.
Reverse Causality: Higher crime rates in specific areas lead to a higher demand for, and concentration of, police forces.
Confounding Factors (X): Economic conditions, state capacity, and urban density can simultaneously influence both policing levels and crime rates.
Draca et al. (2011) - "Panic on the Streets of London":
Studied whether more police reduce crime using the July 7, 2005, London Tube bombings as an exogenous shock.
Operation Theseus: Following the attacks, police activity in central London increased by over for six weeks. This provided a natural experiment to study policing levels independent of local crime trends.
Spurious Correlations
Data can show correlations that appear significant but have no causal link:
Example 1: The distance between Neptune and Mercury correlates with petroleum consumption in Azerbaijan (r = 0.798, r^2 = 0.636, p < 0.01).
Example 2: Total wind power generated in Taiwan correlates with Google searches for "I am tired" (r = 0.980, r^2 = 0.961, p < 0.01).
Chocolate and Nobel Prizes (Messerli, 2012 NEJM): A surprisingly powerful correlation was found between per capita chocolate intake and the number of Nobel laureates. Messerli humorously suggested chocolate improves cognitive function, though these findings are only hypothesis-generating and require randomized trials.
Course Methodology and Logistics
Core Problems of Causal Inference:
Selection Bias.
Reverse Causality.
Omitted Variables / Confounders.
Spurious Correlations.
Research Designs to be Covered:
Controlled Experiments.
Natural (Quasi) Experiments.
Difference-in-Differences.
Regression Discontinuity.
Instrumental Variables.
Evaluation Criteria:
Final Exam: ( minutes).
Two Problem Sets: (done in groups of or ; involves Stata programming and replication of academic papers).
Weekly Quizzes: (online via MyARIEL).
Student Requirements:
Pre-requisites: Microeconomics, Econometrics, and foundations of Stata programming.
Assignments: Bring a newspaper article starting Week 2 (April 8) that makes a causal claim for critical evaluation.
Readings: "Mastering 'Metrics" by Angrist and Pischke (2014) and "Causal Inference: The Mixtape" by Scott Cunningham (2021).
Final Takeaway: Data alone, without assumptions or specific research designs, cannot answer causal questions. Policy decisions require understanding what happens if an intervention occurs, especially as resources are scarce.