Week 10: One-way Repeated-Measure ANOVA I

Week 10 Overview

• This week's lecture covers:
  • Stats "map" for this week
  • Research design
  • Statistical concept
  • Variance partitioning
  • Sums of squares
  • Omnibus F summary table (in Stata)
  • Omnibus effect size (in Stata)
  • Assumptions
  • Compound symmetry and sphericity
  • Adjustment to F-test when compound symmetry or sphericity is not met
  • A complete repeated-measure one-way ANOVA demonstration in Stata

Analyses in Design & Statistics II: One-way within-subjects (or repeated-measures) ANOVA

• Research design:
  • Same participants undergo different conditions or are tested at different times.
• Statistical concept:
  • Similar to other ANOVAs but with a different error term.
• Numerical DV and categorical IV.
• One IV.
• Two or more levels.
• Related Groups.

Research Design

• Variables: 1 numerical DV + 1 categorical IV (or factor), with more than two conditions.
• Research Design (from a sampling perspective): Within-subjects (repeated) groups (similar to the paired t-test).
• One-way Within-subjects ANOVA: Same Group, Different Conditions/Times.
• One-way Between-subjects ANOVA: Different Groups.

Potential Issues and Solutions

• Advantages of a within-subjects design:
  • Fewer participants.
  • In repeated-measures ANOVA analyses, we remove the naturally occurring differences associated with the participants from the error or residual term. Because $F = MS(effect)/MS(error)$, enhancing the analysis's power.
• Potential issue with the within-subjects design? Order (carry-over) effect.
• Solutions (to distribute/cancel the carry-over effect)?
  • Randomise the order of conditions.
  • Counterbalance the order.
• Counterbalance the order (assuming we have 3 conditions):
  • Participant 1: Condition 1 >> Condition 2 >> Condition 3
  • Participant 2: Condition 2 >> Condition 3 >> Condition 1
  • Participant 3: Condition 3 >> Condition 1 >> Condition 2
  • Participant 4: Condition 1 >> Condition 2 >> Condition 3
  • Participant 5: Condition 2 >> Condition 3 >> Condition 1
  • Participant 6: Condition 3 >> Condition 1 >> Condition 2

Research Design Contd.

• But sometimes, we just can't randomise or counterbalance the order.
  • E.g., Participants' anxiety level before, immediately after, and a week after a public speech.
• Research Design (regarding the IV-DV relationship)
  • Observing naturally occurring changes over time/conditions, e.g., the example above, or in longitudinal studies >> quasi-experimental design.
  • When the assignment of order/conditions can be manipulated >> experimental design.
  • Repeated-measures ANOVA can be used in quasi-experimental or experimental designs, depending on the control researchers have over the conditions.
• Very common in experimental psychology (e.g., cognitive psychology).

Research Design Example

• Abstract: Pain is a protective perceptual response shaped by contextual, psychological, and sensory inputs that suggest danger to the body. Sensory cues suggesting that a body part is moving toward a painful position may credibly signal the threat and thereby modulate pain.
• Method: We used a within-subjects, randomized, double-blinded, repeated-measures design. The distance from the center position to the left or right position at which participants experienced the onset of pain (i.e., the pain-free range of motion) was quantified in three conditions. Virtual rotation was (a) 20% less than actual physical rotation (rotation gain = 0.8), (b) equal to actual physical rotation (rotation gain = 1), or (c) 20% greater than actual physical rotation (rotation gain = 1.2). The order of the three conditions was counterbalanced across participants.
• To test our main hypothesis (i.e., that visual information that overstates or understates true rotation can affect movement-evoked pain), we compared pain-free range of motion across the three conditions. We used repeated measures analysis of variance (ANOVA) with Bonferroni-corrected pairwise comparisons.

Two-step Analysis as Other ANOVAs

• One-way ANOVA Omnibus F-test
  • Step 1 (this week): Is there any difference among conditions at all?
    • H0: μcond.1 = μcond.2 = μcond.3 = μcond.x … or there's no difference between any conditions whatsoever; it is also called an omnibus null hypothesis.
    • H1: At least one of the means under a condition differs from the rest.
  • Step 2 (next week): Where do the differences emerge?
    • Comparing subsets of conditions.

Statistical Concept

• Variance Partitioning
• Sums of Squares
• Omnibus F Summary Table

Variance Partitioning CF. ONE-WAY BETWEEN-SUBJECTS ANOVA

• One-way between-subjects ANOVA
  • Total Variability = Between-group Variability + Within-group Variability (e.g., subjects' differences and other unexplained).
• One-way within-subjects (repeated-measures) ANOVA
  • Total Variability = Between-condition Variability + Within-subjects Variability.
  • Within-subjects Variability = Between-subjects Variability (e.g., subjects’ differences) + Errors/Residuals (Unexplained).
• $F = Signal/Noise = MS(group)/MS(error)$
• $F = Signal/Noise = MS(condition)/MS(error)$

Variance Partitioning Contd. CF. ONE-WAY BETWEEN-SUBJECTS ANOVA

• One-way between-subjects ANOVA
  • Total Variability = Between-group Variability + Within-group Variability (e.g., subjects' differences and other unexplained)
• One-way within-subjects (repeated-measures) ANOVA
  • Total Variability = Between-condition Variability + Within-subjects Variability
  • Within-subjects Variability = Between-subjects Variability (e.g., subjects’ differences) + Errors/Residuals (Unexplained)
• Partialling effects due to differences associated with participants out of the error term.
• Reduce the noise in the signal-to-noise ratio, and make it a more powerful statistical test.

Separation of Variability in the DV ALL OBSERVATIONS

• Next, we'll use this very simple example (5 participants encountered all 3 conditions, resulting in 15 observations in total) to demonstrate:
  • Total variability in the DV
  • Between-subjects variability (variability due to naturally occurring individual differences)
  • Within-subjects variability
    • Condition variability (variability explained by conditional/ treatment differences)
    • Error variability (unexplained variability)
• There will be some equations for demonstration purposes, but you don't need to remember or hand-calculate them.
• Example Data:
  • Participants: Participant 1, Participant 2, Participant 3, Participant 4, Participant 5
  • Condition 1: 80, 90, 50, 40, 45; Condition Mean: 61
  • Condition 2: 70, 80, 45, 35, 30; Condition Mean: 52
  • Condition 3: 60, 70, 40, 30, 30; Condition Mean: 46
  • Participant Mean: 70, 80, 45, 35, 35
  • Grand Mean: 53

Total Variability

• TOTAL SUMS OF SQUARES
• Total Sums of Squares or SSTotal or SS(total): Total variability in the DV
• $SSTotal = \sigma(x<em>{ij} - x</em>{grand})^2$
• $SSTotal = (80-53)^2 + (90-53)^2 + (50-53)^2 + (40-53)^2 + (45-53)^2 + (70-53)^2 + (80-53)^2 + (45-53)^2 + (35-53)^2 + (30-53)^2 + (60-53)^2 + (70-53)^2 + (40-53)^2 + (30-53)^2 + (30-53)^2 = 5840$

Variability Between Participants

• SUBJECTS SUM OF SQUARES
• Subjects Sums of Squares, or SSsubjects, or SS(subjects): Variability due to differences associated with participants themselves.
• $SSsubjects = k \sigma(x<em>i - x</em>{grand})^2$
• $SSsubjects = 3 \times ((70-53)^2 + (80-53)^2 + (45-53)^2 + (35-53)^2 + (35-53)^2) = 5190$

Variability Between Conditions

• CONDITION SUM OF SQUARES
• Condition Sums of Squares, or SSCondition, or SS(condition): Variability due to differences induced by our conditions or treatments (the effect we are interested in, and want to examine).
• $SSCondition = n \sigma(x<em>i - x</em>{grand})^2$
• $SSCondition = 5 \times ((61-53)^2 + (52-53)^2 + (46-53)^2) = 570$

Unexplained Variability

• ERROR/RESIDUAL SUM OF SQUARES
• Error (Residual) Sums of Squares, or SSerror/residual, or SS(error/residual): Within-subjects variability that cannot be explained by the condition or treatment differences.
• $SSError = SSTotal – SSSubjects – SSCondition$
• $SSError = 5840 – 5190 – 570 = 80$

F Omnibus Summary Table

• anova DV Condition Participants, repeated(Condition).
• The way we interpret the F summary table is mostly the same as that of a between-subjects one-way ANOVA.

F Omnibus Summary Table Contd.

• The way we interpret the F summary tablet is mostly the same as that of a between-subject one-way ANOVA.
• $df(condition) = k – 1 = 3 – 1 = 2$, where k = #levels.
• $df(participants) = n – 1 = 5 – 1 = 4$, where n = #participants.
• $df(residual) = (k – 1)(n – 1) = 2 \times 4 = 8$
• $df(total) = kn – 1 = 3 \times 5 – 1 = 14$, where kn = total # observations.
• $df(total) = df(condition) + df(participants) + df(residual)$
• $MS = SS/df$

F Omnibus Summary Table Contd.

• TEST STATISTIC: F VALUE
• The way we interpret the F summary tablet is mostly the same as that of a between-subject one-way ANOVA.
• For one-way repeated-measures ANOVAs:
  • $F = MS(effect)/MS(residual)$
  • $F(Condition) = 285/10 = 28.5$
  • $F(Participants) = 1297.5/10 = 129.75$
• Effect of interest: $F(2, 8) = 28.50$, p < .001

Omnibus Effect Size

• Partial eta-squared is typically used:
  • $\eta_p^2 = SS(Condition)/[SS(Condition)+SS(Residual)]$
  • $\eta_p^2 = 570/(570+80) = 0.87692308$
• Cohen's (d) effect magnitude: small ≈ .2; medium ≈ .5; large > .8

Assumptions

• Numeric DV: DV is measured on an interval/ratio scale.
• Related groups, but independent observations: The same participants are present in all groups (or, across the groups, participants are related to each other in some ways).
• Normality: DV are normally distributed at each level of the IV.
• Compound symmetry or sphericity: Variances of the differences between all combinations of related groups are equal.
  • Would be ensured by design and sampling.

Data Format

• Long format: DV in a single column, with repeating participants and conditions.
• Wide format: DV under each condition in a column, with non-repeating participants.
• Use it for running the actual ANOVA analysis.
• Use for assumptions checking, esp. compound symmetry or sphericity.
• reshape wide DV, i(ID) j(IV)
• reshape long DV, i(ID) j(IV)

Assumption 3: Normality

• FOR LONG FORMAT
  • by Condition, sort: swilk DV
  • histogram DV, by(Condition)

Assumption 3: Normality Contd.

• FOR WIDE FORMAT
  • swilk DV1 DV2 DV3
  • histogram DV1, name(dv1)
  • histogram DV2, name(dv2)
  • histogram DV3, name(dv3)
  • graph combine dv1 dv2 dv3, cols(3)

Assumption 4: Compound Symmetry

• Compound symmetry is met when:
  • Variances are equal across conditions: Similar to the equal variance assumption in one-way between-subjects ANOVA (i.e., equal variance across levels).
  • Covariances (correlations) between conditions are also equal/constant (…because we are measuring the same participants!): Pairwise correlations between conditions are the same for all pairs of conditions.
• summarize DV1 DV2 DV3
• pwcorr DV1 DV2 DV3

Assumption 4: Compound Symmetry Contd.

• Compound symmetry is met when:
  • Variances are equal across conditions: Similar to the equal variance assumption in one-way between-subjects ANOVA (i.e., equal variance across levels).
  • Covariances (correlations) between conditions are also equal/constant (…because we are measuring the same participants!): Pairwise correlations between conditions are the same for all pairs of conditions.
• Can use a formal statistical test for compound symmetry: Lawley's test.
• What we want here: p > .05
• mvtest correlations DV1 DV2 DV3

Sphericity

• Sphericity is a less restrictive case of compound symmetry.
• Sphericity requires that all of the variances of pairwise differences between conditions are equal.
• gen diff2v1 = DV2-DV1
• gen diff3v2 = DV3-DV2
• gen diff3v1 = DV3-DV1
• summarize diff2v1 diff3v2 diff3v1

Sphericity

• Sphericity is a less restrictive case of compound symmetry.
• Sphericity requires that all of the variances of pairwise differences between conditions are equal.
• Formal statistical test for Sphericity? Mauchly's test.
• Unfortunately, Mauchly's test is not implemented in Stata  …
• Even better news: There are also ways to adjust the F-test even if the compound symmetry or sphericity is not met ☺

Adjust the F Test?

• Smaller df1 >> larger p
• Larger df1 >> smaller p

Epsilon Adjustments

• The alternative F tests adjust df1 (condition df) by multiplying it by an epsilon constant, making df1 smaller.
  • If sphericity or compound symmetry is met (i.e., epsilon = 1), no need to adjust.
  • More severe violations require reductions in df1 (i.e., epsilon < 1).
  • The value of the F-statistic is not changed, but the p-value has

Epsilon Adjustments Contd.

• Three different methods to adjust:
  • The Box epsilon (AKA the lower bound) is the worst-case value
    • Its epsilon = 1 / (k – 1), where k = #levels in the IV
    • Here, we have 3 levels, epsilon = 1/(3-1) = 0.5
  • Greenhouse-Geisser epsilon is less conservative, ranges between 1/k-1 to 1
    • When the G-G epsilon > 0.75, it could be too conservative, and the Huynh-Feldt may be more appropriate
  • Huynh-Feldt is the least conservative, used when G-G epsilon > 0.75
  • The potential severity of the adjustment increases as the number of conditions increases!

Research Design & Statistics Steps

• For all the statistical analyses we'll talk about in this unit (regression, ANOVA, non-parametric analyses), we'll follow a standard process:
  • Before getting into the data, we must understand (design steps):
    1. Our research questions and hypotheses we are trying to answer with our data.
    2. Our sampling population.
    3. How our variables measured (type and scale).
  • Then, getting into the data analysis, we then (statistics steps):
    4. Describe variables using appropriate UNIVARIATE numeric and graphical summaries
    5. Describe variables using appropriate BIVARIATE numeric and graphical summaries
    6. Formally test assumptions
    7. Fit appropriate statistical model(s)
       1. Omnibus F-test
       2. Follow-up analyses
    8. Interpret results + draw conclusions

A Simple Example

• Variables:
  • One numerical DV – number of injuries in a three-month period
  • One categorical IV – 3 types of costumes children wear: Mickey vs. Superman vs. Batman.
  • Design – every child in the sample wears all three costume types, each for 3 months.
• Research question & hypothesis:
  • Research question: Does wearing different types of costumes lead to different injuries for children?
  • Research Hypothesis: Children will sustain more injuries during the periods in which they wear superhero costumes (Superman and Batman) compared to when they wear a non-superhero character costume (Mickey).
• Statistical Analysis? One-way repeated-measures ANOVA

Bivariate Numeric & Graphical Summaries

• tabstat DV, by(IV) stat(n mean sd)
• cibar DV, over(IV)

Assumptions

• Numeric DV: DV is measured on an interval/ratio scale
• Related groups, but independent observations: The same participants are present in all groups (or, across the groups, participants are related to each other in some ways)
• Normality: DV are normally distributed at each level of the IV
• Compound symmetry or sphericity: Variances of the differences between all combinations of related groups are equal

Normality Assumption

• NOTE. LONG DATA FORMAT IS USED HERE.
• histogram DV, by(IV)
• by IV, sort: swilk DV

Compound Symmetry or Sphericity

• First, let's reshape the data into a wide format
• reshape wide injury, i(id) j(cos)

Compound Symmetry or Sphericity Contd.

• Next, the formal statistical test for compound symmetry or sphericity? Do we need to perform additional statistical tests for sphericity? If so, which test should we use?
• mvtest correlations injury1 injury2 injury3

Compound Symmetry or Sphericity Contd.

• Compound symmetry
  • Pairwise correlations across conditions are the same
  • Equal variance across conditions
• Sphericity
  • Variances of pairwise differences between conditions are equal
  1. Generate the differences
  2. Summarize the differences
• OTHER WAYS TO CHECK THIS ASSUMPTION?

Running Statistical Analysis

• First, let's reshape the data (AGAIN) from a wide format to a long format
• reshape long injury, i(id) j(cos)

Running Statistical Analysis Contd.

• Stata command
• anova DV IV ID, repeated (IV)
• The compound symmetry assumption has been met (e.g., from Lawley's test), so we can ignore the second part of the output.
• F(2, 18) = 4.38, p = .028

Detour 1: What if the compound symmetry assumption is violated?

• F(1.85, 18) = 4.38, p = .032
• df1 = 2 x 0.9260 = 1.852

Detour 2: Compare with the one-way between-subjects ANOVA

• Residual being separated into two parts
• Same effect SS, but larger F in repeated

Effect Size

• Cohen's (d) effect magnitude: small ≈ .2; medium ≈ .5; large > .8
• Large-sized effect: $\eta_p^2 = .33$

Write Up

• A one-way repeated measures ANOVA was conducted to examine how children wearing different costumes may lead to varying levels of injuries over a three-month period. The results showed that costume type elicited statistically significant and large differences in the mean frequency of injuries, F(2, 18) = 4.38, p = 0.028, $\eta_p^2 = .33$

Conclusions

• Unlike between-subjects ANOVA, the within-subjects (repeated-measures) design employs the same group of participants, either at different points in time or under different conditions.
• Statistically, repeated-measures one-way ANOVA has more power than its between-subjects counterpart because the variability associated with individual participants has been moved out of the error term. The F-value computed is the ratio of between-condition variance vs. error variance.
• It shares some assumptions with the between-subjects one-way ANOVA: 1) a numerical DV; 2) independent observations; however, repeated subjects; 3) normally distributed DV by conditions. However, the repeated-measures one-way ANOVA also has a unique assumption, sphericity or compound symmetry.
• There are ways to adjust the p-values even when the sphericity or compound symmetry assumption is not met; the p-value is adjusted by applying an epsilon to the degrees of freedom.
• One-way repeated-measure ANOVA is also a two-step process for its analysis—the first step tests whether any one group differs from the others at all (omnibus test); and the second step follows it up to discover where the differences arise.

Lecture Learning Outcomes

• After this week's lecture, you know:
  • What one-way repeated-measures ANOVA analysis is, and how it is similar to and different from its between-subjects counterpart
  • What kinds of research questions and research designs are suitable for repeated-measures one-way ANOVA
  • How is the variance in repeated-measures ANOVA partitioned, and how does that differ from its between-subjects counterpart
  • The assumptions of the repeated-measures one-way ANOVA
  • The ANOVA summary table and how to interpret it
• In Stata, you should be able to:
  • Open data files
  • Test assumptions of repeated-measures one-way ANOVA, especially the sphericity or compound symmetry assumption
  • Run a one-way ANOVA analysis
  • Understand which p-value to use depending on the sphericity or compound symmetry assumption test results
  • Create and save a .do file for your commands (syntax)