Randomized Experimental Designs

Comprehensive practice flashcards covering the theory of random assignment, internal validity threats, types of control groups, experimental designs with and without pretests, factorial designs, blocking/stratification, and individual versus cluster randomization.

How does random assignment differ fundamentally from random sampling?

Random assignment is a procedure where each unit has a nonzero probability of being assigned to a condition to make experimental comparison groups similar at baseline. Random sampling is the random selection of units from a population to ensure the sample is representative of that population.

Which specific threat to internal validity is uniquely prevented by random assignment?

Selection bias. Other threats to internal validity still operate, but randomization distributes them equally among groups so that events affecting the outcome are no more likely to occur in the treatment group than in the control group.

Why does random assignment not guarantee that groups will have perfectly equal baseline pretest means?

Due to sampling error (the "luck of the draw"). Randomization equates groups on the statistical expectation of equal means across both measured and unmeasured variables over multiple randomizations, providing an unbiased estimate rather than identical values in any single trial.

How does the theory of randomization explain the relationship between condition assignment and confounding variables?

It ensures that the pretest correlation between condition assignment and confounding variables is not significantly different from zero, making it unlikely that extraneous variables are confounded with the treatment condition.

What are the common types of control conditions used in randomized experimental designs?

1. Current practice (standard treatment)

2. Dose-response (multiple variants in treatment strength)

3. Parallel treatment / Attention controls (an intervention not hypothesized to affect the outcome)

4. Placebo

5. Wait-list (controls receive the intervention prior to the end of the study)

What are the main methodological limitations of a posttest-only randomized experimental design ($R \quad X \quad O$ / $R \quad O$)?

Because there is no baseline assessment, it cannot measure participant attrition, the magnitude of change, statistical regression to the mean, maturation, or history effects.

What advantages does including a pretest provide in a randomized experimental design ($R \quad O \quad X \quad O$)?

It allows researchers to evaluate and adjust for attrition, analyze both within-group and between-group effects, and maximize statistical power by allowing participants to serve as their own controls.

Why is the "observe then randomize" design ($O \quad R \quad X \quad O$ / $O \quad R \quad O$) considered methodologically stronger than randomizing prior to the pretest observation?

Gathering baseline observations before randomization prevents post-randomization dropout, expectation effects, or participant reactions to condition assignment from biasing the pretest measures.

What is a factorial design, and why is it considered more efficient than conducting separate single-factor experiments?

A factorial design investigates two or more independent variables (factors), each with two or more discrete levels (such as a $2 \times 2$ design). It allows simultaneous testing of main effects and interaction effects while requiring fewer participants because individuals do "double duty" across conditions.

What is the difference between a main effect and an interaction effect in a factorial design?

A main effect is the average effect of an independent variable on the dependent variable across all other levels of factors in the study. An interaction effect occurs when the effect of one independent variable varies depending on the level of another independent variable (a multiplicative rather than additive effect).

In a $2 \times 2$ factorial design plot of group means, what do parallel lines between levels indicate?

Parallel lines indicate the presence of main effects without an interaction effect, meaning the effect of Factor B is consistent and additive across the levels of Factor A.

In a $2 \times 2$ factorial design plot of group means, what do non-parallel lines with different slopes indicate?

Non-parallel lines indicate an interaction effect, demonstrating that the impact of Factor B on the outcome varies depending on the level of Factor A.

What is blocking in experimental design, and what is its primary statistical benefit?

Blocking involves dividing the sample into homogeneous subgroups based on a covariate associated with the outcome (e.g., age or clinical subtype) and then randomizing participants within each block. This ensures that variability within each block is smaller than the variability across the entire sample.

What are the primary benefits and limitations of individual-level randomization?

Benefits: Permits analysis of treatment effects within higher-order units (e.g., clinics), reduces the likelihood of baseline covariate imbalance, and maximizes statistical power. Limitations: Risks treatment contamination among individuals within the same setting and can be administratively difficult to deliver different treatments in the same cluster.

What are the main advantages and methodological challenges of group (cluster) randomization?

Advantages: Administrative simplicity, lower delivery costs, prevention of contamination within intact groups (e.g., schools, worksites), and suitability for group-level interventions. Challenges: Dependencies among units within clusters violate independence assumptions and elevate Type I error rates (false alarms), matching higher-order units can fail to ensure balance, and ethical consent dilemmas arise.