RCT Quiz

Faulty randomization

When the treatment and control groups do not look the same, this is likely an issue with ___

Partial compliance

When some people don’t take up treatment

What attrition in an RCT can affect

• Balance

• Whether you can get at the “real” causal effect

• Generalizability

What affects the generalizability of an RCT?

• Who agrees to be in the trial

• Who gets the intervention

• Who is included in the trial

Why do we care about RCTs?

• Gold standard - everything tries to replicate a RCT

• Good at reducing threats to internal validity

  • Although there are still threats to IV/EV

• Ethical issues are trickier

When should we use a RCT?

• You’re certain about the direction and magnitude of an intervention effect

• You can conduct an ethical evaluation

• You are piloting a new approach

• You know the treatment does not harm

True or False: RCTs are experiments

True

True or False: RCTs can have multiple treatments or approaches

True — these are called treatment arms (greenwashing & trash pick up vs trash pick up only)

RCT Types

• Pretest-posttest control group design

• Posttest only control group design

• Cross-over design

• Delayed intervention design

Placebo

Beneficial effect of the intervention on the group that did not get it

Double blind

Means researchers don’t know who is in treatment or control (placebo is administered).
Pretty uncommon in policy settings. Very common in medicine

Ethical issues in RCTs

• Is it ethical to withhold a program from a group of people?

• Is it ethical to randomly determine the treatment that people will receive?

• Is it ethical to expose people to experimental treatments or programs?

• Is it ethical to invest time/resources into RCTs

What makes an RCT different ethically?

• Potential to answer policy-relevant questions by cleanly identifying causal impact (which may be very important)

• Collection of (potentially sensitive) primary data on individuals and communities (may be costly)

• Close collaboration between researchers and implementers (may influence research/introduce bias)

• Randomization itself, which may change how programs are delivered.

Causality Risks and Benefits

The better the ability to answer causally and the more important a question is to designing effective policy, the higher the benefit for a given risk.

• Just because there may be a lot of benefit – some risks are just not ok.

Often RCTs are very focused/too small scale/too specific to be very useful.

• If data collected can’t be used widely why put the burden on research subjects?

Close Collaboration w/ Programs

• Researchers can affect the design of programs if they are close to it which might have ethical concerns

• Programs, unlike research, are not regulated

• Researchers could potentially influence how programs are implemented which may have ethical concerns (ie: deciding to randomize a program or randomizing who gets a finite resource)

Ethical concerns with randomization

• Someone doesn’t get a treatment — if treatment is life saving we may have to delay it because it’s unethical otherwise

• Always winners and losers with randomization

True or False: No research has ethical concerns.

False. All research has ethical concerns, especially RCTs.

Types of Ethical Issues

• Knowingly causing harm

• Choice to participate or not (informed consent — allowing people to opt out)