Advanced Statistics: Repeated Measures ANOVA and Post-Hoc Analysis for Clinical Therapy

Case Study: Depression Therapy and Repeated Measures ANOVA

  • Context and Data Source: The analysis centers on an example from page 395 of the textbook, focusing on depression scores across different time points. This is a practical application of the Repeated Measures Analysis of Variance (ANOVA).
  • The Utility of ANOVA: While a t-test can compare two points (e.g., before and after), ANOVA is required when there are three or more groups or time points. In clinical research, this allows for the evaluation of therapy longevity.
  • Experimental Design: The factor is Time, and there are k=3k = 3 time points:
    • Before Therapy: Baseline depression level.
    • After Therapy: Depression level immediately following the conclusion of treatment.
    • Six-Month Follow-Up: Depression level six months after treatment to determine if the therapeutic effects "stick."
  • Participants: The study includes n=5n = 5 participants, each measured at every time point (N=15N = 15 total scores).
  • The Problem of Efficacy: A therapy is deemed truly successful not just if depression drops immediately, but if it stays low. If depression rebounds to original levels after treatment ends, the therapy's success is limited as it only works while the patient is actively in it.

Preliminary Calculations (Step 0)

  • Provided Values: In an exam or lab (such as Lab 3.3), certain values like the Sum of Squares (SSSS) for each group are often provided to save time. For this problem:
    • SS1SS_{1} (Before): 273.2273.2
    • SS2SS_{2} (After): given in the context of calculations.
    • SS3SS_{3} (6-Month): given in the context of calculations.
  • Participant Totals (PP): A new element in Repeated Measures is the PP value, which is the sum of scores for a single participant across all time points.
    • P1=179P_1 = 179
    • P2=151P_2 = 151
    • P3=191P_3 = 191
    • P4=187P_4 = 187
    • P5=185P_5 = 185
  • Treatment Totals (TT):
    • T1=373T_1 = 373
    • T2=258T_2 = 258
    • T3=261T_3 = 261
  • Grand Total (GG): The sum of all scores. This can be calculated in two ways to check for errors:
    • G=T1+T2+T3=892G = T_1 + T_2 + T_3 = 892
    • G=P1+P2+P3+P4+P5=892G = P_1 + P_2 + P_3 + P_4 + P_5 = 892
  • Calculation of Means (MM):
    • M1=T1n=3735=74.6M_1 = \frac{T_1}{n} = \frac{373}{5} = 74.6
    • M2=51.6M_2 = 51.6
    • M3=52.2M_3 = 52.2
  • Identifying Severity: Comparing the PP values identifies which individuals have the most severe depression across the study. In this data, Person 3 exhibits the highest total scores (191191), while Person 2 has the mildest depression (151151).

Hypothesis Testing (Steps 1-4)

  • Step 1: Hypotheses:
    • Null Hypothesis (H0H_0): Time spent in therapy has no effect on depression scores (μ1=μ2=μ3\mu_1 = \mu_2 = \mu_3).
    • Alternative Hypothesis (H1H_1): Time spent in therapy has an effect on depression scores; at least one average score is different.
  • Step 2: The ANOVA Table: In Repeated Measures, the "Within-Treatments" variance is split into two components: Participants (individual differences) and Error (random noise).
    • SS Between: 1718.531718.53
    • SS Participants: 446.4446.4
    • SS Error: 98.898.8
    • SS Total: 2263.732263.73
    • Degrees of Freedom (dfdf):
      • dfbetween=k−1=2df_{between} = k - 1 = 2
      • dftotal=N−1=14df_{total} = N - 1 = 14
      • dfwithin=N−k=12df_{within} = N - k = 12
      • dfparticipants=n−1=4df_{participants} = n - 1 = 4
      • dferror=(k−1)(n−1)=8df_{error} = (k - 1)(n - 1) = 8
    • Mean Squares (MSMS):
      • MSbetween=SSbetweendfbetween=1718.532=859.27MS_{between} = \frac{SS_{between}}{df_{between}} = \frac{1718.53}{2} = 859.27
      • MSerror=SSerrordferror=98.88=12.35MS_{error} = \frac{SS_{error}}{df_{error}} = \frac{98.8}{8} = 12.35
    • F-Obtained: F=MSbetweenMSerror=859.2712.35=69.58F = \frac{MS_{between}}{MS_{error}} = \frac{859.27}{12.35} = 69.58
  • Step 3: F-Critical: Using the table for α=0.05\alpha = 0.05 with dfnumerator=2df_{numerator} = 2 and dfdenominator=8df_{denominator} = 8, the critical value is 4.464.46.
  • Step 4: Decision: Since the obtained FF (69.5869.58) is substantially larger than the critical value (4.464.46), we reject the null hypothesis. There is a significant effect of therapy over time.

Effect Size and APA Documentation (Step 5)

  • Eta Squared (η2\eta^2): This measures the proportion of variance explained by the therapy. In Repeated Measures, individual differences (SSparticipantsSS_{participants}) are removed from the denominator to get a cleaner look at the treatment effect.
    • η2=SSbetweenSSbetween+SSerror\eta^2 = \frac{SS_{between}}{SS_{between} + SS_{error}}
    • η2=1718.531718.53+98.8=0.946\eta^2 = \frac{1718.53}{1718.53 + 98.8} = 0.946
  • Interpretation: The effect size is roughly 95%95\%. This is considered "absurdly large" in real-world psychology (where 25%25\% is the threshold for a large effect), but serves as a clear indicator of a strong relationship in this word problem.
  • APA Style Summary: F(2,8)=69.58,p<0.05,η2=0.95F(2, 8) = 69.58, p < 0.05, \eta^2 = 0.95

Pairwise Comparisons (Step 6: Tukey’s HSD)

  • Calculating Tukey's HSD: This determines the minimum difference required between two means to be considered statistically significant.
    • q-Value: Found using k=3k = 3 and dferror=8df_{error} = 8. At the 0.050.05 level, q=4.04q = 4.04.
    • HSD Formula: HSD=q×MSerrorn=4.04×12.355=6.35HSD = q \times \frac{\sqrt{MS_{error}}}{n} = 4.04 \times \frac{\sqrt{12.35}}{5} = 6.35
  • Pairwise Results:
    • Before vs. After: ∣74.6−51.6∣=23.0|74.6 - 51.6| = 23.0 (Significant, as 23.0>6.3523.0 > 6.35).
    • Before vs. 6-Month: ∣74.6−52.2∣=22.4|74.6 - 52.2| = 22.4 (Significant, as 22.4>6.3522.4 > 6.35).
    • After vs. 6-Month: ∣51.6−52.2∣=0.6|51.6 - 52.2| = 0.6 (Not Significant, as 0.6<6.350.6 < 6.35).

Interpretation and Pattern Analysis

  • Success Visualization: In this study, the ideal success pattern is a sharp decrease in depression scores that remains low over time.
    • Observed Pattern: Depression dropped from 74.674.6 to 51.651.6 and remained essentially flat at the follow-up (52.252.2).
    • Clinical Conclusion: The therapy is successful. It caused a significant reduction in depression that was maintained six months after treatment concluded. The slight increase of 0.60.6 points in the follow-up is not statistically significant and can be attributed to random noise (sampling error).
  • Participant Variability: By using a Repeated Measures design, researchers can distinguish between variance caused by the therapy and variance caused by individual differences in baseline depression (e.g., Person 3 being more depressed than Person 2 regardless of treatment).