DR

Post Hoc Tests for Repeated Measures ANOVA

Post Hoc Tests for Repeated Measures ANOVA

  • After rejecting the null hypothesis in a repeated measures ANOVA, post hoc tests are needed to determine where the significant differences lie among the population means.

Potential Explanations for Significant Differences

  • When a repeated measures ANOVA indicates a significant difference, several explanations are possible:
    • No difference between pre-treatment and one-month post-treatment, but a significant difference at six months post-treatment due to delayed treatment effects.
    • One month post-treatment is significantly different from both pre-treatment and six months post-treatment, indicating a temporary effect.
    • Pre-treatment scores are significantly different from both one-month and six-month post-treatment scores, suggesting a sustained treatment effect.
    • All three time points (pre-treatment, one month, and six months) are significantly different from each other, indicating continuous change over time.

Importance of Identifying the Correct Explanation

  • Determining the correct explanation is crucial for understanding the treatment's effectiveness and making informed decisions about future treatment modifications.

Pairwise Comparisons

  • Post hoc tests involve making pairwise comparisons to identify specific differences between time points.
  • For the example, the following comparisons are needed:
    • Pre-treatment vs. one month post-treatment
    • Pre-treatment vs. six months post-treatment
    • One month post-treatment vs. six months post-treatment

Tukey's Honestly Significant Difference (HSD) Test

  • Tukey's HSD test is used for pairwise comparisons in repeated measures ANOVA.
  • The formula is similar to the one used in independent samples ANOVA, but with MS error in the numerator instead of MS within treatments.
  • Formula: HSD = q \sqrt{\frac{MS_{error}}{n}}, where:
    • q is the q value from the Tukey's HSD table.
    • MS_{error} is the mean square error from the ANOVA.
    • n is the sample size in each condition.

Determining the Q Value

  • To find the q value, you need:
    • K: the number of conditions (e.g., 3 for pre-treatment, one month, and six months).
    • Degrees of freedom for error: the denominator of the F ratio (MS error).
    • Alpha level (e.g., 0.05).
  • The Q table is similar to the F table but contains different values and is not interchangeable.

Example Calculation

  • Given: Three conditions, four people in each condition, and degrees of freedom for error = 6, alpha = 0.05.
  • From the Q table, the Q value q = 4.34.
  • The calculated HSD threshold is 2.014.

Pairwise Comparisons and Interpretation

  • Calculate the mean differences for each pairwise comparison.
  • Compare the absolute value of each difference to the HSD threshold.

Example Pairwise Comparisons

  • Comparison 1: Before vs. One Month After
    • The means are 7 and 3.
    • Difference: |7 - 3| = 4.
    • 4 > 2.014, so there is a significant difference.
  • Comparison 2: Before vs. Six Months After
    • The means are 7 and 3.25.
    • Difference: |7 - 3.25| = 3.75.
    • 3.75 > 2.014, so there is a significant difference.
  • Comparison 3: One Month After vs. Six Months After
    • The means are 3 and 3.25.
    • Difference: |3 - 3.25| = 0.25.
    • 0.25 < 2.014, so there is no significant difference.

Three-Part Interpretation

  • Integrate the context, independent variable, and dependent variable into the interpretation:
    • People are significantly less anxious one month after cognitive behavioral therapy compared to before therapy.
    • People are significantly less anxious six months after cognitive behavioral therapy compared to before therapy.
    • There is no significant difference in people's anxiety levels between one month and six months after cognitive behavioral therapy.