CR3 Experiment

Research Strategies (RS)

Descriptive
Correlational/Non-experimental
Quasi-experimental
Experimental

Descriptive (RS)

Describes what is present in the population, no causal claims

Correlational/Non-experimental (RS)

no

Quasi-experimental (RS)

No random assignment, but we measure the DV before the manipulation, more control than correlational but less than true experiment

Experimental (RS)

Researcher manipulates an IV, then randomly assigns participants to conditions

The 5 Elements of a True Experiment

Manipulation
Measurement
Comparison
Control
Random Assignment

Manipulation

The researcher creates the independent variable by intervening, guaranteeing chronological order

Measurement

Measuring the dependent variable, and checking the manipulation actually worked

Comparison

There must be at least two conditions to compare, without comparison you cannot attribute any change to the manipulation

Control

Holding potential confounds constant

Random Assignment

Randomly placing participants into groups by chance

Confounds

A variable that changes alongside your manipulation, making it impossible to tell whether it was your manipulation or the other variable that caused the observed effect

Internal Validity

Can we attribute any observed difference to the manipulation, and only the manipulation?

External Validity

Do conclusions hold outside of the experiment?

Population Validity

Can we generalise across people?

Ecological Validity

Does the experiment situation reflect real life?

Between Subjects Design

Different participants in different conditions

Single Factor Design

A single factor [Factor Name]: [Level 1] vs [Level 2] Between/Within subjects design

Factorial Design

A 2 ([Factor 1]: [Level 1] vs [Level 2]) x 2 ([Factor 2]: [Level 1] vs [Level 2]) Between/Within subjects design

Noise (Random Error)

Individual differences that vary randomly across participants

Bias (Systematic Error)

Individual differences that are systematically distributed between conditions, creates a confound

Threats to Internal Validity (Between Subjects)

Selection Bias
Differential Attrition
Diffusion 
Compensatory Equalisation
Compensatory Rivalry
Resentful Demoralisation

Selection Bias

Non random assignment means the groups participant characteristics differ before the experiment begins

Differential Attrition

Participants drop out more from one condition than others, changes group composition over time, undermining comparability.

Diffusion

Control group participants learn about or adopt the treatment through communication with treatment group participants.

Compensatory Equalisation

Providing the control group with compensatory treatment as it feels unfair to withhold

Compensatory Rivalry

Control group knows they are the underdog so work harder to prove it doesn't matter. Artificially reduces the treatment effect.

Resentful Demoralisation

Control group performs worse than normal because they resent not getting treatment

Interaction Effects

when the effect of one factor depends on the level of another factor

Interaction Effect Visualisation

Parallel lines = No interaction

Non parallel lines = Interaction

Manipulation Checks 

Participation Check
Attention Check
Perception Check

Participation Check

Did the participant actually encounter the manipulation

Attention Check

Did the participant follow instructions correctly

Perception Check

Did the participant perceive the manipulation as intended

Research Ethics

Informed Consent & Debriefing

Informed Consent 

Before an experiment participants must consent, after being informed what will happen, potential risks, and that participation is voluntary

Debriefing

After participation full disclosure must be given, participants have the right to know what they participated in.

Within Subjects Design

Each participant experiences all conditions.

Within Subjects Pros

No bias from individual differences

No noise from individual differences 

Requires fewer participants (higher statistical power)

Within Subjects Cons

Time related threats to internal validity

Order effects

Demand characteristics

Contrast effects

Threats to Internal Validity (Within Subjects)

History
Maturation
Instrumentation
Regression to the mean
Carry Over Effects
Demand Characteristics

History (Threat to Within Subjects)

An external event occurs between measurements, the event not the manipulation could explain the difference

Maturation (Threat to Within Subjects)

Natural changes over time: fatigue, hunger, boredom, practice. These affect performance on the second condition systematically

Instrumentation (Threat to Within Subjects)

the measurement instrument or interviewer changes overtime

Regression to the mean (Threat to Within Subjects)

Extreme scores at time 1 tend to move toward the average at time 2

Carry Over Effects (Threat to Within Subjects)

Experiencing condition A creates a lasting change that affects responding in condition B.

Demand Characteristics (Threat to Within Subjects)

Participants who experience both conditions may figure out the study's hypothesis and change their behaviour accordingly.

Counterbalancing

Distributes order effects equally across conditions by varying the order

Symmetrical Carry Over

The order effect is the same regardless of which treatment comes first

Non Symmetrical Carry Over

The order produces different effects depending on which treatment is first

Randomisation Checks

test whether conditions differ significantly on demographic and control variables before the manipulation

Correlational/Non-experimental Design

No assignment, participants self select into conditions based on a measured variable

Non-experimental Design Formula

X O (or O alone)

Quasi-experimental Design

No random assignment, but we measure the DV before the manipulation

Quasi-experimental Design Formula

O X O (pre test / manipulation / post test)

Combined Strategy Designs

Mixed Design & Combined Strategy

Mixed Design

Uses both within subject and between subject factors

Combined Strategy

Uses both manipulated and measured factors