ECON 2310 Experiments and Quasi-Experiments

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/68

encourage image

There's no tags or description

Looks like no tags are added yet.

69 Terms

1
New cards

experiments

Potential outcomes, causal effects, and idealized experiments

– Threats to validity of experiments (internal / external)

– Application: Experimental estimates of the effect of class size reductions

2
New cards

quasi-experiments

Examples, diff-in-diff, IV, RDD

– Potential problems with quasi-experiments

– Experimental and quasi-experimental estimates in heterogeneous populations

3
New cards

why study experiments?

Ideal randomized controlled experiments provide a conceptual benchmark for assessing observational studies.

• Actual experiments are rare ($$$) but influential.

• Experiments can overcome the threats to internal validity of observational studies. But they have their own threats to internal and external validity.

4
New cards

• Thinking about experiments helps us to understand quasi-experiments, or

“natural experiments”. They provide “natural” variation generated by “as if ” random assignment.

5
New cards

an experiment is

designed and implemented consciously by human researchers. An experiment randomly assigns subjects to treatment and control groups.

– Example: clinical drug trials

6
New cards

A quasi-experiment or natural experiment has

a source of randomization that is “as if ” randomly assigned, but this variation was not the result of an explicit randomized treatment and control design.

– Example: natural disasters

7
New cards

program evaluation

the field of statistics aimed at evaluating the effect of a program or policy.

– Example: a job training program

8
New cards

example: Clinical drug trial: does a proposed drug lower cholesterol?

Y = cholesterol level

X = treatment or control group (or dose of drug)

9
New cards

example: job training


10
New cards

example: class size effect


11
New cards

A treatment has a causal effect for

a given individual:

give the individual the treatment and something happens → this result is (possibly) different than that if she does not get the treatment.

12
New cards

potential outcome

the outcome for an individual under a potential treatment or potential non-treatment

For an individual, the causal effect is the difference in potential outcomes if you do or don’t get the treatment.

13
New cards

An individual’s causal effect cannot be observed because

you can give the subject the treatment, or not − but not both! Why?

– Because there are no identical two individuals

14
New cards

estimating treatment effect in idealized experiments


15
New cards

the homogeneous treatment effect is

B1


16
New cards

if Xi is randomly assigned, then


17
New cards

differences estimator


18
New cards

additional regressors background


19
New cards

Two reasons to include control variables in a regression analysis of the effect of a randomly assigned treatment


20
New cards

more on reason 2: what if probability of assignment depends on Wi ?



21
New cards

conditional mean independence is satisfied if


22
New cards

randomization based on covariates example: men and women are assigned to course on dancing, but women assigned with higher probability and have better dancing skills than men prior to course. because of this…


23
New cards

randomization based on covariates: based on example Xi is randomly assigned given Wi so


24
New cards

randomization based on covariates: OLS estimator of B1 in regression


25
New cards

causal effect in previous example might depend on


26
New cards

use interactions


27
New cards

threat to internal validity: failure to randomize


28
New cards

thread to internal validity: failure to follow treatment protocol


29
New cards

treat to internal validity: attrition


30
New cards

threat to internal validity: experimental effects


31
New cards

threats to external validity


32
New cards

Experimental Estimates of Effect of Class Size Reductions: Project STAR background


33
New cards

deviations from design: project STAR


34
New cards

effect of class size reductions: results


35
New cards

Effect of Class Size Reductions: Adding Covariates


36
New cards

Effect of Class Size Reductions: Comparison with Observational Studies


37
New cards

recap on why study experiments


38
New cards

quasi experiment/natural experiment


39
New cards

2 types of quasi experiments

treatment X

variable Z

40
New cards

treatment X


41
New cards

variable Z


42
New cards

differences in differences estimator


43
New cards

assigning DID variables


44
New cards

DID graph


45
New cards

differences regression formulation


46
New cards

The differences-in-differences estimator allows for

systematic differences in pre-treatment characteristics, which can happen in a quasi-experiment because treatment is not randomly assigned.

47
New cards

DID with control variables


48
New cards

why include control variables?


49
New cards

IV estimation


50
New cards

one example of IV estimation


51
New cards

regression discontinuity estimators background


52
New cards

In sharp regression discontinuity design,

everyone above (or below) the threshold w0 gets treatment.

53
New cards

In fuzzy regression discontinuity design,

crossing the threshold w0 influences the probability of treatment, but that probability is between 0 and 1.

54
New cards

sharp regression discontinuity


55
New cards

sharp: If crossing the threshold affects 𝑌𝑖 only through the treatment


56
New cards

sharp regression discontinuity design in picture


57
New cards

slopes could be

differetn on each side of discontinuity


58
New cards

fuzzy regression discontinuity


59
New cards

If crossing the threshold has no direct effect on 𝑌𝑖,


60
New cards

fuzzy regression discontinuity example


61
New cards

heterogenous population


62
New cards

in potential outcome terminology of het population


63
New cards

in general B1i of het population


64
New cards

het populations person i’s treatment effect


65
New cards

What if the source of variation is unobserved? This raises two questions:


66
New cards

what do we want to estimate

most commonly avg treatment effect

but can also be effect of treatment on treated

67
New cards

average treatment effect


68
New cards

effect of treatment on treated


69
New cards

OLS with heterogeneity and random assignment