CR1 Lecture 10: Experiments

Correlation

An association between two variables where changes in one tend to occur with changes in the other.

Causation

When a change in X produces a change in Y.

Three criteria for causation

Covariation, chronology and non-spuriousness.

Covariation

X and Y must vary together or show an association.

Chronology

The cause (X) must occur before the effect (Y).

Non-spuriousness

Alternative explanations for the relationship between X and Y must be ruled out.

Why correlation does not prove causation

Correlation alone cannot establish time order or exclude alternative explanations.

Spurious relationship

An apparent association between X and Y that can be explained by a common cause.

Common cause

A third variable (C) that influences both X and Y.

Mediation

X influences a mediator (M), which then influences Y.

Spuriousness vs mediation

Spuriousness involves C causing both X and Y; mediation involves X causing M, which influences Y.

Experiment

A research method where researchers manipulate X to examine its effect on Y.

When to use an experiment

When testing whether X causes Y and manipulation is feasible and ethical.

Experimental research question example

Does exposure to negative political news decrease political trust?

Experimental hypothesis example

Students exposed to negative news will report lower political trust than those in the control group.

Manipulation

Deliberately changing an independent variable to examine its effect.

Independent variable (X)

The variable manipulated or treated as the possible cause.

Dependent variable (Y)

The measured outcome that may be affected by X.

Internal validity

Confidence that the manipulation caused the observed outcome rather than another factor.

External validity

The extent to which findings generalize to other people, settings or situations.

Experimental design

The organization of an experiment to investigate whether X causes Y.

Control group

A comparison group that does not receive the experimental treatment.

Experimental group

A group that receives the manipulation or treatment.

Random assignment (R)

Randomly allocating participants to experimental conditions.

Manipulation symbol (X)

Indicates where the treatment occurs.

Observation symbol (O)

Indicates where an outcome is measured.

Pretest (O₁)

A measurement taken before treatment.

Posttest (O₂)

A measurement taken after treatment.

One-group pretest-posttest design

O₁ → X → O₂. One group is measured before and after treatment, without a control group.

Weakness of one-group pretest-posttest

Alternative explanations such as maturation and repeated testing cannot be adequately ruled out.

When to use one-group pretest-posttest

When measuring change within one group, although it provides weak causal evidence.

Two-group pretest-posttest design

Experimental

Purpose of two-group pretest-posttest

Compares changes over time between treatment and control groups.

Two-group posttest-only design

Experimental

Advantage of posttest-only design

Avoids repeated testing effects caused by a pretest.

When to use posttest-only design

When a pretest could influence the outcome, especially in short-term experiments.

Solomon four-group design

Group 1

Purpose of Solomon four-group design

Tests treatment effects, pretest effects and whether pretesting changes the treatment effect.

Disadvantage of Solomon four-group design

Requires more participants and resources, making it expensive.

True experiment

An experiment that uses random assignment to conditions.

Self-selection

Participants choose which experimental condition they enter.

Selection bias

Systematic differences between groups caused by the selection process.

Self-selection vs selection bias

Self-selection can cause selection bias by creating systematically different groups.

Purpose of random assignment

Prevents self-selection and reduces systematic differences between experimental groups.

Does random assignment guarantee equal groups?

No, groups are comparable on average across repeated random assignments, not necessarily in one study.

Why random assignment improves internal validity

It reduces the likelihood that other characteristics systematically determine treatment assignment.

Random sampling

Randomly selecting participants from a population to improve representativeness.

Random sampling vs random assignment

Sampling determines who enters the study; assignment determines which condition they enter.

Maturation

Natural changes in participants over time. Solution

Repeated testing

A pretest influences later responses. Solution

Self-selection threat

Participants choose their experimental condition. Solution

Selection bias threat

Groups differ systematically in relevant characteristics. Solution

Experimenter bias

Researchers' expectations influence their behaviour and potentially the results. Solution

Demand effect

Participants change behaviour because they think they know the hypothesis. Solution

Instrumentation

Differences in measurement methods influence outcomes. Solution

Diffusion

Treatment information spreads to the control group. Solution

Attrition

Participants drop out or have missing outcome data. Solution

External events

Events outside the experiment influence the outcome. Solution

Double-blind design

Participants and relevant researchers do not know treatment assignments, reducing experimenter bias and demand effects.

Maturation vs external events

Maturation involves natural changes within participants; external events occur outside the experiment.

Experimenter bias vs demand effect

Experimenter bias comes from researchers' expectations; demand effects come from participants' expectations.

Why control groups reduce maturation threats

Both groups experience similar natural changes, helping isolate the treatment effect.

Between-subjects design

Different participants experience different experimental conditions.

Within-subjects design

The same participants experience multiple experimental conditions.

Within-subjects limitation

Order and carryover effects may influence responses to later conditions.

Factorial design

An experiment examining combinations of two or more independent variables.

Full factorial design

Includes every combination of the independent variables' conditions.

Moderation

When the effect of one independent variable on Y depends on another variable.

How to recognize moderation

The effect of X on Y differs across levels of another variable.

How experimental control affects external validity

Highly controlled settings may be less representative of real-world situations.

How field experiments affect validity

Realistic settings may improve generalizability but reduce control over outside influences.

Internal vs external validity

Internal validity concerns causal conclusions; external validity concerns generalizability.

How to improve internal validity

Use random assignment, comparable control groups, standardized procedures and blinding where possible.

How to improve external validity

Use representative samples and realistic research settings where appropriate.

Why experiments can establish causality

Manipulation, time order, random assignment and control of alternatives strengthen causal inference.