Partial Compliance in RCTs

pubpol 639 program evaluation

what is partial compliance

situations in which members of the treatment or control group do not “comply” with their assignment

ex: individuals in treatment group do not receive any treatment or do not complete any treatment or individuals in comparison group receive treatment

control crossover

individuals in comparison group receives treatment

why is partial compliance not always a mistake or problem

• many interventions or treatments are not mandatory and individuals can be offered the opportunity to participate but are not meant to be forced

why is partial compliance a problem

non-compliance reduces the difference in program exposure between T and C groups which will bias the estimated treatment effect toward zero. to calculate the treatment effect you need to calculate the average outcome of all members in the T group. including those who didn’t receive the treatment and the avg will be attenuated if only a portion were treated

how to minimize control crossover

• monitor participants closely

• randomize at higher level

how to deal with partial compliance

can’t ignore the T members who do not receive treatment or count them as part of the C group because it introduces bias - treatment take-up is likely endogenous and would make T not random

notation for whether an invidiual was assigned to treatment

Z_i

notation for whether an individual received the treatment

D_i

what are the different “types” in an RCT

compliers, always-takers, never-takers, defiers

compliers

do what they are told (calculate last)

always takers

always do the treatment even if not assigned. for people assigned to the treatment group, can’t distinguish between always takers and compliers so we calculate always-takers as people in the control group who receive treatment

never-takers

never do the treatment even if assigned. for the control group, can’t distinguish never-takers from compliers so we calculate people assigned to treatment group who didn’t take treatment

defiers

do the opposite of what they are told. can’t calcuate

ITT

intent-to-treat → the effect of being assigned to treatment.

• combines effect of treatment itself with probability of take-up

• assumes independence and SUTVA

• also referred to as the reduced form effect

• estimated via OLS regression

• ITT is often the most policy-relevant parameter

LATE

Local Average Treatment Effect

• the effect of the intervention on the compliers

  • “local” refers to the fact that we are measuring the effect for a specific group of individuals who are on the margin of participating and encouragement or opportunity will push them into receiving treatment

calculating LATE

divide it by the difference in treatment rates between the assigned T and C groups
LATE = ITT/FS

FS

“fist-stage” → the difference in treatment participation

2SLS

two-stage least squares → used to estimate the LATE
type of instrumental variables estimation and allows inclusion of covariates and calculation of s.e.

assumptions required to identify ITT

1. independence - no selection bias

2. SUTVA - no spillovers

assumptions required to identify LATE

1. independence - no selection bias

2. SUTVA - no spillovers

3. Relevance

4. Excludability (exclusion restriction)

   1. Monotonicity

exclusion restriction

the randomized treatment assignment does not influence the outcome direction, that is, other than through the fact that it increases the likelihood the individual will receive the treatment

ex: winning a charter school admissions lottery cannot influence student achievement by making the child feel proud about winning something

Monotonicity

the assumption rules out the existence of defiers

ITT vs LATE

ITT is the effect of of intention to treat so it includes everyone who was assigned to treatment group. LATE captures the treatment effect for the compliers

TOT

Treatment-on-the-Treater - weighted average of the effect on the compliers and the always-takers.
If there are no always-takers, then LATE=TOT
one cannot choose to calculate TOT vs LATE, if there are crossovers in your study you should interpret the treatment effect estimates as LATEs

Calculating Y̅_T-Y̅_C

If you calculate Y̅_T-Y̅_C in an RCT with perfect compliance, you can interpret this as the ATE which in some cases is ATT and if you calculate in an RCT with imperfect compliance you interpret this as ITT