5.1 pre-experiment

5.1 pre-experiment

Pre-experimental Design (O₁ X O₂)

• O₁ refers to observation of a group, or pretest

• X refers to a treatment

• O₂ refers to the second observation, or posttest

O₁ = Pretest → X = Treatment → O₂ = Posttest

In pre-experimental design

• There is no control group and no random assignment.

• You measure one group before and after a treatment.

Why is a Pre-Experimental Design considered a bad experiment?

• Any change from O₁ to O₂ cannot be assumed to be caused by the treatment, because many threats to internal validity (e.g., maturation, history, practice effects) could explain the difference.
  → Results are inconclusive; cannot establish cause & effect.

NONEQUIVALENT CONTROL GROUP DESIGN

• treatment group & comparison group are compared using pretest and posttest measures

• No random assignment → groups are pre-existing: (e.g., two classrooms, two clinics)

Main Limitation of the Nonequivalent Control Group Design

• considered groups are not randomly assigned, they may differ before treatment, so any difference at post-test could be due to:

• The treatment

• Pre-existing group differences (participant variables)

→ Lower internal validity (cannot be sure the treatment caused the change).

What threats to internal validity are controlled in a Nonequivalent Control Group Design and why?

• History

• Maturation

• Testing effects

• Instrumentation changes

• Regression to the mean

→ both groups take pretest & posttest = impossible to explain differences between groups

What internal validity threats cannot be controlled in a Nonequivalent Control Group Design and why?

• Selection × Maturation

• Selection × History

• Selection × Instrumentation

• Differential regression

• → Pre-existing group differences remain a possible alternative explanation.

Selection x Maturation (internal validity)

• when individuals in one group grow more experience → Additive effect of selection & maturation occurs

• threat →

• -  1. self-selected samples

• -  2. when the comparison group comes from another treatment group.

Selection x History (internal validity)

• event other than the treatment affects one group & not the other

• (e.g. Local history effects)

Selection x Instrumentation (internal validity)

• changes in a measuring instrument are more likely to be detected in one group than they are in another.

Differential statistical regression (internal validity)

• Differential regression → occur when regression is more likely in one group than in another.

EXTERNAL VALIDITY IN NONEQUIVALENT CONTROL GROUP DESIGN

• results may apply only to that specific group, setting, or time → because groups are pre-existing,
  → ““To establish external validity, the study must be replicated with different participants & settings.””

Key Limitation for External Validity

• not possible to answer everything about a hypothesis.

• EV requires looking at patterns of results across multiple studies, not relying on a single finding.