1/68
Looks like no tags are added yet.
experiments
Potential outcomes, causal effects, and idealized experiments
– Threats to validity of experiments (internal / external)
– Application: Experimental estimates of the effect of class size reductions
quasi-experiments
Examples, diff-in-diff, IV, RDD
– Potential problems with quasi-experiments
– Experimental and quasi-experimental estimates in heterogeneous populations
why study experiments?
Ideal randomized controlled experiments provide a conceptual benchmark for assessing observational studies.
• Actual experiments are rare ($$$) but influential.
• Experiments can overcome the threats to internal validity of observational studies. But they have their own threats to internal and external validity.
• Thinking about experiments helps us to understand quasi-experiments, or
“natural experiments”. They provide “natural” variation generated by “as if ” random assignment.
an experiment is
designed and implemented consciously by human researchers. An experiment randomly assigns subjects to treatment and control groups.
– Example: clinical drug trials
A quasi-experiment or natural experiment has
a source of randomization that is “as if ” randomly assigned, but this variation was not the result of an explicit randomized treatment and control design.
– Example: natural disasters
program evaluation
the field of statistics aimed at evaluating the effect of a program or policy.
– Example: a job training program
example: Clinical drug trial: does a proposed drug lower cholesterol?
Y = cholesterol level
X = treatment or control group (or dose of drug)
example: job training

example: class size effect

A treatment has a causal effect for
a given individual:
give the individual the treatment and something happens → this result is (possibly) different than that if she does not get the treatment.
potential outcome
the outcome for an individual under a potential treatment or potential non-treatment
For an individual, the causal effect is the difference in potential outcomes if you do or don’t get the treatment.
An individual’s causal effect cannot be observed because
you can give the subject the treatment, or not − but not both! Why?
– Because there are no identical two individuals
estimating treatment effect in idealized experiments

the homogeneous treatment effect is
B1

if Xi is randomly assigned, then

differences estimator

additional regressors background

Two reasons to include control variables in a regression analysis of the effect of a randomly assigned treatment

more on reason 2: what if probability of assignment depends on Wi ?


conditional mean independence is satisfied if

randomization based on covariates example: men and women are assigned to course on dancing, but women assigned with higher probability and have better dancing skills than men prior to course. because of this…

randomization based on covariates: based on example Xi is randomly assigned given Wi so

randomization based on covariates: OLS estimator of B1 in regression

causal effect in previous example might depend on

use interactions

threat to internal validity: failure to randomize

thread to internal validity: failure to follow treatment protocol

treat to internal validity: attrition

threat to internal validity: experimental effects

threats to external validity

Experimental Estimates of Effect of Class Size Reductions: Project STAR background

deviations from design: project STAR

effect of class size reductions: results

Effect of Class Size Reductions: Adding Covariates

Effect of Class Size Reductions: Comparison with Observational Studies

recap on why study experiments

quasi experiment/natural experiment

2 types of quasi experiments
treatment X
variable Z
treatment X

variable Z

differences in differences estimator

assigning DID variables

DID graph

differences regression formulation

The differences-in-differences estimator allows for
systematic differences in pre-treatment characteristics, which can happen in a quasi-experiment because treatment is not randomly assigned.
DID with control variables

why include control variables?

IV estimation

one example of IV estimation

regression discontinuity estimators background

In sharp regression discontinuity design,
everyone above (or below) the threshold w0 gets treatment.
In fuzzy regression discontinuity design,
crossing the threshold w0 influences the probability of treatment, but that probability is between 0 and 1.
sharp regression discontinuity

sharp: If crossing the threshold affects 𝑌𝑖 only through the treatment

sharp regression discontinuity design in picture

slopes could be
differetn on each side of discontinuity

fuzzy regression discontinuity

If crossing the threshold has no direct effect on 𝑌𝑖,

fuzzy regression discontinuity example

heterogenous population

in potential outcome terminology of het population

in general B1i of het population

het populations person i’s treatment effect

What if the source of variation is unobserved? This raises two questions:

what do we want to estimate
most commonly avg treatment effect
but can also be effect of treatment on treated
average treatment effect

effect of treatment on treated

OLS with heterogeneity and random assignment
