Post Hoc Tests for Repeated Measures ANOVA

Post Hoc Tests for Repeated Measures ANOVA

• After rejecting the null hypothesis in a repeated measures ANOVA, post hoc tests are needed to determine where the significant differences lie among the population means.

Potential Explanations for Significant Differences

• When a repeated measures ANOVA indicates a significant difference, several explanations are possible:
  • No difference between pre-treatment and one-month post-treatment, but a significant difference at six months post-treatment due to delayed treatment effects.
  • One month post-treatment is significantly different from both pre-treatment and six months post-treatment, indicating a temporary effect.
  • Pre-treatment scores are significantly different from both one-month and six-month post-treatment scores, suggesting a sustained treatment effect.
  • All three time points (pre-treatment, one month, and six months) are significantly different from each other, indicating continuous change over time.

Importance of Identifying the Correct Explanation

• Determining the correct explanation is crucial for understanding the treatment's effectiveness and making informed decisions about future treatment modifications.

Pairwise Comparisons

• Post hoc tests involve making pairwise comparisons to identify specific differences between time points.
• For the example, the following comparisons are needed:
  • Pre-treatment vs. one month post-treatment
  • Pre-treatment vs. six months post-treatment
  • One month post-treatment vs. six months post-treatment

Tukey's Honestly Significant Difference (HSD) Test

• Tukey's HSD test is used for pairwise comparisons in repeated measures ANOVA.
• The formula is similar to the one used in independent samples ANOVA, but with MS error in the numerator instead of MS within treatments.
• Formula: HSD = q \sqrt{\frac{MS_{error}}{n}}, where:
  • q is the q value from the Tukey's HSD table.
  • MS_{error} is the mean square error from the ANOVA.
  • n is the sample size in each condition.

Determining the Q Value

• To find the q value, you need:
  • K: the number of conditions (e.g., 3 for pre-treatment, one month, and six months).
  • Degrees of freedom for error: the denominator of the F ratio (MS error).
  • Alpha level (e.g., 0.05).
• The Q table is similar to the F table but contains different values and is not interchangeable.

Example Calculation

• Given: Three conditions, four people in each condition, and degrees of freedom for error = 6, alpha = 0.05.
• From the Q table, the Q value q = 4.34.
• The calculated HSD threshold is 2.014.

Pairwise Comparisons and Interpretation

• Calculate the mean differences for each pairwise comparison.
• Compare the absolute value of each difference to the HSD threshold.

Example Pairwise Comparisons

• Comparison 1: Before vs. One Month After
  • The means are 7 and 3.
  • Difference: |7 - 3| = 4.
  • 4 > 2.014, so there is a significant difference.
• Comparison 2: Before vs. Six Months After
  • The means are 7 and 3.25.
  • Difference: |7 - 3.25| = 3.75.
  • 3.75 > 2.014, so there is a significant difference.
• Comparison 3: One Month After vs. Six Months After
  • The means are 3 and 3.25.
  • Difference: |3 - 3.25| = 0.25.
  • 0.25 < 2.014, so there is no significant difference.

Three-Part Interpretation

• Integrate the context, independent variable, and dependent variable into the interpretation:
  • People are significantly less anxious one month after cognitive behavioral therapy compared to before therapy.
  • People are significantly less anxious six months after cognitive behavioral therapy compared to before therapy.
  • There is no significant difference in people's anxiety levels between one month and six months after cognitive behavioral therapy.