Repeated-Measures ANOVA Notes

Repeated-Measures ANOVA (Within-Subjects ANOVA)

Learning Objectives

• Describe repeated-measures ANOVA (within-subjects ANOVA).

• Understand the rationale of repeated-measures ANOVA, including its benefits and potential issues.

• Conduct and interpret ANOVA.

• Report ANOVA results.

Experimental Designs

• Between-subjects design: Two or more independent groups of subjects are tested (e.g., males vs. females, depressed vs. controls, young vs. old).

• Repeated-measures design (aka within-subjects design): The same subjects are tested under all experimental conditions.

  • Example: One group of muppets receives treatment 1, then treatment 2, and then treatment 3.

Example of Between-Subjects Experiment

• Research question: Does weather affect math problem solving?

• 27 undergrads assigned to 1 of 3 conditions (groups):

  • Raining outside

  • Snowing outside

  • Sunny outside

• Subjects took 30 minutes to solve problems.

• H0 (Null Hypothesis): No significant difference in the number of problems solved between the 3 conditions.

• H1 (Alternative Hypothesis): There will be at least two means that differ significantly.

Variability Between Subjects

• In between-subjects designs, random sampling aims to cancel out individual characteristics across groups.

Repeated-Measures ANOVA: Rationale

• Individual differences should matter less because the same people are tested in all conditions.

Repeated ANOVA: Sources of Variance

1. Variance created by the experimental manipulation.

2. Variance created by individual differences is 'cancelled out' by repeatedly testing the same participants.

Benefits of Repeated Measures Designs

• Increased sensitivity: Error variance is reduced by 'cancelling out' individual differences.

• Economy: Fewer participants are needed.

• Counteracting Fatigue/Learning: Randomization and counterbalancing are used to mitigate fatigue or learning effects.

Repeated Measure ANOVA Example

• Research question: Effect of weather on problem-solving.

• IV (Independent Variable): Weather conditions

• DV (Dependent Variable): Number of problems solved

• Design: One-way repeated measures ANOVA

• H0: No significant difference in number of problems solved in 3 conditions.

• H1: At least one pair of conditions will differ.

Partitioning the Variance

• Analogy: It's all about the cake!

Partitioning Variability for Between-Subjects ANOVA

1. Total variability

2. Between-group variability

3. Within-group (error) variability

Partitioning Variability for Repeated-Measures ANOVA

• Mean differences reflect:

  • Experimental manipulation (weather)

  • Error

• Error here is variability within each person, not between people.

• Subject 1’s performance change between conditions is partly due to treatment and partly due to 'natural' variability.

• The goal is to 'cancel out' this 'natural' variability.

Within-Subjects Variability (SS_{within-subjects})

• Calculate variability of each subject around their own mean and sum them all up.

• SS{within-subjects} = SS{treatment} + SS_{residual (error)}

• Determining if there is more treatment variability or 'error' variability in SS_{within-subjects}.

Treatment (Model) Variability (SS_{treatment})

• Variability of treatment means from the grand mean.

The Repeated-Measures ANOVA Ratio

• SS{model} and SS{residual} are adjusted by df (degrees of freedom) to produce MS{model} and MS{residual}.

• The df error for repeated measures is (nr. treatments – 1)*(nr. participants – 1).

• Calculate the F-ratio as usual.

Comparing to Between-Subjects ANOVA

• Unsystematic (residual or error) variability = individual differences + differences in how someone behaves at different times.

• Individual differences are a factor in between-subjects ANOVA but not in repeated-measures ANOVA.

• Residual variability 'should' be lower in repeated-measures ANOVA.

• SS_{model} is the same (variation of the three treatment means around grand mean).

• The error (or residual) variance is different (smaller for the repeated-measures).

• Reducing the denominator of the F-ratio increases the F-ratio.

Assumptions in Repeated-Measures ANOVA

• Sphericity assumption: "The variance of the difference across pairs of conditions (treatments) should be almost the same."

• Assessed using Mauchly’s test.

  • p < .05: Sphericity is violated.

  • p > .05: Sphericity is not violated.

What is Sphericity?

• Is the variance in the difference columns too different?

• Mauchly’s test determines if the difference is too different.

• At least 3 columns (i.e., conditions) are needed to compare the variance of these differences.

Mauchly’s Test

• "Mauchly's Test of Sphericity indicated that the assumption of sphericity has not been violated, \chi^2(2) = 3.35, p = .19."

• Mauchly’s test has a \chi^2 distribution.

Corrections for Violation of Sphericity Assumption

• Greenhouse-Geisser

• Huynh-Fieldt

• Estimate

• These corrections rely on adjusting the degrees of freedom.

Repeated Measures ANOVA: Main Output Table

• Greenhouse-Geisser is a correction used when the sphericity assumption is not met (Mauchly’s test p < .05).

• This correction 'corrects' the degrees of freedom.

Interpreting Results

• If F-value is significant, at least one pair of means should be different.

Posthocs for Repeated-Measures ANOVA

• Some of the familiar post-hoc tests are not valid with repeated-measures ANOVA.

Posthoc Tests in Repeated Measures ANOVA

• Compare all pairs of means.

• Some posthocs are more conservative than others.

  • Less 'powerful'

  • Less 'sensitive'

  • More likely to result in a 'false-negative' error.

• The Bonferroni test is more conservative than the Holm test.

• Many posthocs are based on a t-test but with the p-value adjusted to penalize the number of tests carried out.

Repeated Measures Design ANOVA with Two IVs

• Example: Test participants on both easy and difficult problems under three weather conditions.

• IV1: weather

• IV2: problem difficulty

• DV: number of problems solved
  *Note that participants contribute scores to each experimental condition.

Repeated Measures ANOVA: Another Example

• Each column represents a combination of weather condition and task’s difficulty.

Table: Tests of repeated-measures Effects

• Factor names are tested

• Test of Sphericity: Cannot perform sphericity tests because there are only two levels of the RM factor, or because the SSP matrix is singular.

Interaction Term

• What is this “Weather * Problem Difficulty ” term and why is it significant?

• An interaction term indicates that the effect of weather significantly depends on how difficult the problems were.

Readings

Corresponding book chapters/pages