insh6300 - week 9 (Quasi-Experimental Methods, Natural Experiments, and Randomized Experiments)

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/104

flashcard set

Earn XP

Description and Tags

Comprehensive vocabulary flashcards covering randomized experiments, natural experiments, quasi-experimental designs, threats to internal and external validity, econometric models, and landmark empirical replications.

105 Terms

1
New cards

Quasi-experiment

A research design used to estimate a treatment effect based on a credible comparison when treatment is not randomly or as-if randomly assigned.

2
New cards

Natural experiment

An observational study where a process outside the researcher's control randomly or as-if randomly assigns units to conditions.

3
New cards

As-if random assignment

A condition in which treatment is not actively randomized by a researcher, but unit characteristics do not influence which group a unit is assigned to.

4
New cards

Comparison group

A group of untreated units used for evaluation in non-randomized studies, commonly used in place of 'control group' when assignment is not randomized.

5
New cards

Discrete treatment

A treatment condition where units receive one of a fixed number of conditions, either two (e.g., pill or placebo) or several distinct alternatives.

6
New cards

Continuous treatment

A treatment condition where units receive varying numerical quantities of an intervention, such as inches of rainfall, years of education, or months of prison sentence.

7
New cards

Dose-response relationship

The mathematical or empirical pattern describing how an outcome changes as the amount or intensity of a continuous treatment increases.

8
New cards

Exogenous treatment

A treatment determined by factors external and unrelated to other causes of the outcome, ensuring treated and untreated units remain comparable.

9
New cards

Endogenous treatment

A treatment determined by factors related to other causes of the outcome or the outcome itself, resulting in non-comparable treated and untreated units.

10
New cards

Selection bias (source of endogeneity)

A threat to causal inference occurring when individuals or units choose treatment conditions for reasons that also correlate with the outcome.

11
New cards

Reverse causality

A form of endogeneity where the outcome variable influences or causes the treatment status, such as a state passing a gun law because gun violence rose.

12
New cards

Confounder

An extraneous variable that simultaneously causes both the treatment and the outcome, creating a spurious statistical association.

13
New cards

Resume audit study (assignment mechanism)

A randomized field experiment where a researcher randomly assigns White- or Black-sounding names to resumes sent to employers to measure hiring discrimination.

14
New cards

Boston charter school lotteries

An institutional natural experiment where oversubscribed public charter schools admit students through random lottery drawings.

15
New cards

New Jersey and Pennsylvania minimum wage policy (1992)

A natural experiment setting where New Jersey raised its minimum wage in 1992 while neighboring Pennsylvania did not, allowing cross-border comparison.

16
New cards

Legal drinking age threshold (age 21)

A sharp legal assignment mechanism where legal access to alcohol begins precisely at age 21, generating an exogenous rule-based boundary.

17
New cards

Kindergarten school entry cutoff (September 1)

An administrative rule assigning children with birthdays before September 1 to start school at age 5, while those born after start at age 6.

18
New cards

Election Day rainfall assignment

A natural assignment mechanism where atypical precipitation on Election Day serves as an exogenous factor reducing voter turnout.

19
New cards

Denmark DNA database policy (May 2005)

A policy change starting in May 2005 where individuals charged with the equivalent of a felony in Denmark could be added to a national DNA database.

20
New cards

Institutional lotteries

Formal procedural lotteries run by organizations—such as the Oregon Medicaid lottery or Vietnam draft lottery—that randomly allocate conditions.

21
New cards

Institutional procedures (natural experiments)

Standard operating procedures of organizations—such as randomly assigning probationers to officers or residents to jury pools—that create as-if random assignment.

22
New cards

Anwar et al. (2012)

A natural experiment investigating the impact of jury race in criminal trials using court software that randomly summons and assigns eligible residents to cases.

23
New cards

Quasi-random (Anwar et al., 2012)

The term used by Anwar et al. because they relied on court software randomization and non-random case scheduling could potentially correlate pool composition with case traits.

24
New cards

Noncompliance in Anwar et al. (2012)

The empirical condition where a Black juror was seated in only 40%40\% of trials with one or more Black pool members, making pool analysis like an ITT with 60%60\% noncompliance.

25
New cards

Exogenous shock

An unanticipated event or process external to a causal model (e.g., natural disasters, policy shifts, or weather) that impacts an independent variable.

26
New cards

Excludability (of an exogenous shock)

The requirement that an exogenous shock affects the outcome strictly through the targeted independent variable, with nothing else changing simultaneously.

27
New cards

Kirk (2009)

A study exploiting Hurricane Katrina as an exogenous shock to examine the causal impact of residential relocation on recidivism among parolees.

28
New cards

Selection on observables

The identifying assumption in regression, matching, and weighting that treatment is as-if randomly assigned once all measured characteristics are accounted for.

29
New cards

Matching

A non-experimental method pairing treated and untreated units with identical or similar measured characteristics to balance observed covariates.

30
New cards

Coarsened exact matching (CEM)

A matching method that temporarily bins continuous covariates into discrete categories to find exact matches between treated and comparison units.

31
New cards

Propensity score matching

A technique that estimates the conditional probability of receiving treatment given observed covariates and matches units on this single predicted score.

32
New cards

Inverse probability of treatment weighting (IPTW)

A statistical method that estimates propensity scores and weights units in a regression model rather than matching them directly.

33
New cards

Within-person fixed effects

A longitudinal modeling strategy that removes all time-invariant differences between units, while still assuming no time-varying confounders and no feedback loops.

34
New cards

Cerda et al. (2012)

A research study cited in lecture titled 'Reducing violence by transforming neighborhoods.'

35
New cards

Interrupted time series (ITS)

A quasi-experimental design using longitudinal data on a single unit before and after an intervention, using the pre-intervention trend as the counterfactual.

36
New cards

Comparative interrupted time series (CITS)

An interrupted time series design that adds an untreated comparison unit to contrast projected trends and control for contemporaneous historical shocks.

37
New cards

Intercept change (in ITS)

The immediate, abrupt level shift in the outcome variable observed at the precise moment an intervention is introduced.

38
New cards

Slope change (in ITS)

The gradual alteration in the trajectory or rate of change of the outcome variable over time following an intervention.

39
New cards

Pennsylvania mandatory minimums study

A comparative ITS study showing that the striking down of mandatory minimums by the Pennsylvania Supreme Court had no statistically significant impact on violent crime compared to Ohio.

40
New cards

Difference-in-differences (DiD)

A quasi-experimental design that calculates treatment effects by comparing the over-time change in a treated group to the over-time change in an untreated comparison group.

41
New cards

Parallel trends assumption

The key identifying assumption in DiD that, in the absence of treatment, the treated and comparison groups would have followed identical outcome trajectories over time.

42
New cards

Average treatment effect on the treated (ATT)

The causal effect estimated by difference-in-differences, representing the mean intervention impact specifically on the units that received treatment.

43
New cards

First difference (in DiD)

The contrast of outcomes before versus after treatment within a group, which nets out time-invariant, stable characteristics of the units.

44
New cards

Second difference (in DiD)

The contrast between the treated group's change and the comparison group's change, which nets out secular time shocks common to all units.

45
New cards

Third difference (Triple Differences)

A design extension comparing a targeted outcome or subgroup to an untargeted one within the same units to remove unit-specific contemporaneous shocks.

46
New cards

DiD numerical estimate calculation

A lecture demonstration where treated change was +73.7+73.7 (67.467.4 to 141.1141.1) and comparison change was +23.6+23.6 (27.427.4 to 51.051.0), yielding an effect of 73.7−23.6=50.173.7 - 23.6 = 50.1.

47
New cards

Synthetic control method (SCM)

A data-driven quasi-experimental method that creates an optimal comparison group by taking a weighted average of untreated units to match a treated unit's pre-treatment trend.

48
New cards

Synthetic control weights

Non-negative weights assigned to comparison units chosen to minimize the discrepancy in pre-intervention outcomes and covariates between the synthetic and treated unit.

49
New cards

Proposition 99 (California)

A 1988 California tobacco control initiative evaluated via SCM against a synthetic control constructed from states like Utah (0.3340.334), Nevada (0.2340.234), and Montana (0.1990.199).

50
New cards

In-time placebo test

A falsification check in SCM or quasi-experiments that artificially re-assigns the intervention date to a pre-treatment period to ensure no false effect appears.

51
New cards

In-space placebo test

A falsification test that iteratively applies the synthetic control method to untreated comparison units to see if the treated unit's estimated effect is an anomaly.

52
New cards

Regression discontinuity (RD)

A quasi-experimental design estimating treatment effects by comparing units located immediately above and below an arbitrary administrative cutoff on a continuous running variable.

53
New cards

Running variable

The continuous rating, score, or covariate (XX) used by a formal rule to assign treatment status based on whether it falls above or below a cutoff.

54
New cards

Sharp regression discontinuity

An RD design where treatment status is a strictly deterministic function of the running variable, jumping from probability 00 to 11 precisely at the threshold.

55
New cards

Fuzzy regression discontinuity

An RD design where crossing the cutoff induces a sharp discontinuity in the probability of treatment rather than deterministic assignment, functioning as an instrumental variable.

56
New cards

Bunching (in RD)

An abnormal concentration of observations clustered just above or below a threshold, signaling that individuals or staff manipulated the running variable.

57
New cards

Latin honors study (RD)

An RD application examining whether employers reward Latin honors (e.g., cum laude) by comparing earnings of graduates immediately on either side of GPA cutoffs.

58
New cards

Tahamont (2019)

A regression discontinuity study evaluating the causal impact of prison security classification levels on serious rules violations in California prisons.

59
New cards

Instrumental variables (IV)

A causal identification method using an exogenous variable (ZZ) to isolate variation in an endogenous treatment (DD) that is unrelated to outcome confounders.

60
New cards

Instrument (ZZ)

A variable that is correlated with the causal variable of interest (DD) but has no direct relationship with the outcome (YY) other than through DD.

61
New cards

Exclusion restriction (in IV)

The fundamental untestable assumption that an instrument (ZZ) affects the dependent variable (YY) solely through its impact on the treatment variable (DD).

62
New cards

Independence assumption (in IV)

The instrumental variable requirement that the instrument (ZZ) is as-if randomly assigned and completely independent of all unobserved confounders (UU).

63
New cards

Same-sex sibling instrument

An instrument exploiting parental preference for mixed-sex children: parents whose first two children share the same sex are more likely to have a third child.

64
New cards

Weak instrument

An instrument that has a low statistical correlation with the endogenous treatment variable (DD), leading to biased IV estimates and large standard errors.

65
New cards

Monotonicity assumption

The IV condition that an instrument shifts units in only one direction relative to treatment—assuming there are compliers, always-takers, and never-takers, but no defiers.

66
New cards

Compliers

The subpopulation of units who take the treatment if encouraged or assigned to do so by the instrument, and who do not take it if not encouraged or assigned.

67
New cards

Local average treatment effect (LATE)

The causal effect estimated by an instrumental variable, representing the average treatment effect specifically for the subgroup of compliers.

68
New cards

Intention-to-treat (ITT) estimate

The estimated causal impact of being offered or assigned to a treatment condition across the entire randomized sample, regardless of actual compliance.

69
New cards

Angrist (2006)

A methodological reference demonstrating how random treatment assignment (ZZ) serves as an instrument for treatment received (DD) under noncompliance.

70
New cards

Loeffler (2013)

An empirical study investigating the causal impact of imprisonment on recidivism and employment using random judicial case assignment as an instrumental variable.

71
New cards

Placebo tests (probing quasi-experiments)

Diagnostic checks assessing outcomes, dates, or cutoffs that could not plausibly be impacted by the intervention to detect hidden confounding.

72
New cards

Sensitivity analyses (in quasi-experiments)

Statistical assessments calculating how strong an unmeasured confounder would have to be to completely explain away an observed treatment effect.

73
New cards

Angrist & Pischke (2009)

Authors of 'Mostly Harmless Econometrics,' a textbook referenced for learning empirical causal inference and quasi-experimental techniques.

74
New cards

Cunningham (2021)

Author of the textbook 'Causal Inference: The Mixtape,' an introductory econometrics text covering quasi-experimental methods.

75
New cards

Random assignment (experimental design)

The experimental procedure that ensures baseline equivalence across treatment conditions on all observed and unobserved pre-treatment confounders by design.

76
New cards

Sample size (experimental ingredient)

A critical design element where larger numbers of units shrink chance pre-treatment baseline imbalances and minimize uncertainty around the estimated ATE.

77
New cards

Treatment effect heterogeneity

The variation in the direction or magnitude of an intervention's effect across different demographic subgroups, individuals, or environmental settings.

78
New cards

Unit compliance

An experimental condition where study participants actually take and adhere to the exact treatment condition to which they were assigned.

79
New cards

Differential attrition

An internal validity threat where participant dropout rates or the characteristics of dropouts differ systematically between treatment and control groups.

80
New cards

Excludability (experimental validity)

The requirement that assignment to a treatment group influences the outcome solely through receipt of the assigned treatment, not through ancillary factors.

81
New cards

Hawthorne effect

An excludability threat where participants alter their behavior simply because they are aware they are being actively studied or observed.

82
New cards

John Henry effect

An excludability violation where control group members exert compensatory effort to compete with the treated group after learning they were denied treatment.

83
New cards

Interference (or contamination)

An experimental breakdown occurring when units assigned to one experimental condition interact with or receive components of another condition.

84
New cards

Marques et al. (1994)

A study where closer parole supervision of treated sex offenders caused higher detection of reoffending rather than true higher recidivism, violating excludability.

85
New cards

Stable Unit Treatment Value Assumption (SUTVA)

The assumption that one unit's treatment assignment does not affect the potential outcomes of other units (no interference) and that treatment dosage is identical.

86
New cards

Compromised randomization

An internal validity threat where experimental assignments are subverted, altered, or bypassed by implementers to determine which subjects receive treatment.

87
New cards

Implementation failure

A breakdown where an intervention is delivered poorly or with inadequate dosage, weakening the treatment without necessarily creating systematic group differences.

88
New cards

Time-varying confounders (in experiments)

Events or external environmental changes occurring during an experiment that differentially affect one experimental condition more than another.

89
New cards

Treatment carryover effects

A validity threat in within-subject designs where exposure to an earlier treatment condition alters or contaminates a subject's response to later treatments.

90
New cards

Order effects

A threat in within-subject experiments where the sequence in which conditions are administered influences the observed outcome.

91
New cards

Floor effect

A measurement limitation where an outcome is so rare or measurement threshold so high that scores cluster at the minimum, obscuring treatment effects.

92
New cards

Ceiling effect

A measurement problem where scores cluster at the maximum possible value, preventing researchers from detecting positive treatment differences.

93
New cards

Sampson (2010)

A study highlighting that nonprobability samples paired with treatment effect heterogeneity cause sample ATEs to diverge from population ATEs.

94
New cards

Ethical dilemma of experimental withholding

The ethical tension between denying a promising treatment to a control group versus the risk of administering untested, potentially harmful interventions.

95
New cards

Staggered approach (experimental rollout)

A deployment strategy for treatments that cannot be delivered simultaneously, where conditions are phased in across randomized cohorts over time.

96
New cards

Excess demand (experimental condition)

A real-world setting where demand for an intervention exceeds capacity, making lotteries or randomized waitlists ethically and practically viable.

97
New cards

Replication failure (in program scaling)

The failure of interventions shown to be effective in single experiments to reproduce benefits when implemented across diverse sites or expanded to larger scales.

98
New cards

Bundled treatments

Multi-component interventions evaluated as a composite package, leaving uncertainty about which specific component drove the observed effect.

99
New cards

Hawaii's HOPE Probation program

A probation program emphasizing swift, certain, and proportionate sanctions; an initial randomized trial found strong positive effects, but multisite replications failed to replicate them.

100
New cards

Judge Alm and Honolulu HOPE implementation

The original judge and setting whose exceptional leadership and master's-level probation staff constituted part of the unique treatment condition that proved difficult to replicate.