Experimental Design and Causality: Quick Notes (Caffeine Reaction Time Lab)

Key Concepts

• Experimental research involves making comparisons between groups; true experiments use random assignment to control groups, while quasi-experiments do not.
• True experiments vs quasi experiments:
  • True: manipulate the independent variable; compare an experimental group to a control group; random assignment used.
  • Quasi: groups are pre-existing or not randomly assigned; harder to infer causation due to confounds.
• Association vs causation:
  • Association: a relationship or correlation between variables (e.g., caffeine and reaction time).
  • Causation requires three conditions: association, temporal precedence, and non-spuriousness.
• Operationalization in the caffeine study: caffeine vs decaf vs nothing measured via reaction time (roller drop task).
• placebo effect: improvement due to expectation rather than the active treatment; controlled with placebo group (decaf) and blinding.
• Blinding: reducing bias by concealing group assignment; single, double, and triple blinding vary who is unaware (participants, experimenters, or data analysts).
• Gold standard: randomized controlled trials (RCTs) with random assignment, placebo controls, and blinding; ideally double blinded.
• Experimental designs:
  • Between groups (between-subjects): different participants in each condition.
  • Within groups (within-subjects, repeated measures): same participants measured under each condition, with a baseline first measurement.
  • Mixed design: combines between and within features.
• Baseline measurement: a pre-treatment measure used for comparison in within-subjects or interrupted time series designs.
• Non-spuriousness: elimination of alternative causes (confounds) via randomization, control groups, blinding.
• Ethical and practical limits: some causal variables (trauma, brain injury) cannot be manipulated; researchers rely on other designs.

Experimental Designs and Their Features

• Between groups design:
  • Two or more separate groups; one receives treatment, others serve as controls.
  • Example: caffeine vs decaf; sometimes include a no-treatment group.
• Within groups design (repeated measures):
  • All participants experience all conditions; measures taken before and after treatment.
  • Advantage: controls for individual differences; disadvantage: practice effects.
• Mixed design:
  • Combines between and within aspects; two measurements (pre and post) across groups.
  • Allows both between-group and within-group comparisons.
• Quasi-experimental designs:
  • Non-equivalent groups or interrupted time series without full random assignment.
  • Stronger threats to internal validity due to confounding group differences.

Randomization and Control of Confounds

• Random assignment:
  • Equal chance of ending up in experimental or control group.
  • Cancels out individual differences (age, motivation, fatigue, etc.) across groups.
  • Essential for eliminating alternative causes (non-spuriousness).
• Why not just randomize to a no-treatment group?
  • To control for placebo effects; use a placebo control (e.g., decaf coffee) to ensure equal experience across groups.
• Placebo controls:
  • Placebo: an intervention with no active ingredient but with expected effect.
  • In psychology/medicine, helps isolate the effect of the active treatment from expectations.
• Blinding:
  • Single blind: participants unaware of their group.
  • Double blind: both participants and experimenters unaware.
  • Triple blind: also analysts/data interpreters unaware.
  • In this study, double blind was used; there was also a test for blinding effectiveness (survey showing ~54% could guess correctly, near chance).

Baseline, Time, and Alternative Explanations

• Baseline measurement:
  • A pre-treatment measurement to compare against post-treatment results.
  • Helps attribute changes to the treatment rather than pre-existing differences.
• Potential confounds (examples discussed):
  • Sleep prior to lab, regular caffeine use, distraction, other substances, time of day, sugar in coffee, ADHD status, age, fatigue, motivation, etc.
• Eliminating confounds:
  • Random assignment distributes confounds evenly.
  • Placebo control accounts for expectancy effects.
  • Blinding reduces biases in treatment administration and data interpretation.

Causation Framework in the Lab

• Hypothesis testing form:
  • If caffeine increases alertness, then $RT<em>{ ext{caffeine}} < RT</em>{ ext{no caffeine}}.$
• Three criteria for causation:
  • Association: caffeine and faster reaction time should be related.
  • Temporal precedence: caffeine exposure precedes the faster reaction time.
  • Non-spuriousness: alternative explanations are ruled out (via randomization, placebo, blinding).
• How the caffeine study fulfills (or falls short of) these:
  • Random assignment helps satisfy non-spuriousness by balancing confounds.
  • Placebo and blinding control for expectation effects.
  • A second control group (no drink) is quasi-experimental, introducing some uncertainty about causality for that group.

Practical Takeaways and Design Variants

• True experiments are the gold standard for establishing causation when feasible, often labeled randomized controlled trials (RCTs).
• Limitations of true experiments: ethical, practical, or cost constraints may prevent manipulation of certain variables.
• Recognize variations:
  • Between groups: compare separate groups.
  • Within groups: compare the same participants across conditions with potential learning effects.
  • Mixed: combine both approaches to balance advantages.
• Ethical note: not all variables can be manipulated; alternative designs are used to infer causal relationships where possible.

Quick Reference: Key Terminology

• Association: correlation between A and B.
• Causation: association + temporal precedence + non-spuriousness.
• Random assignment: equal chance of group allocation; balances confounds.
• Placebo effect: improvement due to expectations, not the active treatment.
• Blinding: concealing group assignment to reduce bias; single/double/triple.
• Between groups vs within groups vs mixed designs.
• Baseline measurement: pre-treatment measure for comparison.
• Gold standard: randomized, placebo-controlled, ideally double-blinded experiments.
• Quasi-experiment: lacks full randomization; more vulnerable to confounds.
• Interrupted time series: quasi analogue where measurements occur before and after an event.
• Reaction time: a common dependent variable for alertness studies; lower is faster.