Bivariate Data Analysis: One-Way ANOVA

T-Test vs. One-Way ANOVA
  • T-Test: Compares means between 22 groups (tt-score).

  • One-Way ANOVA: Compares means across 22 or more groups (FF-score).

  • Familywise Error Rate: Running multiple tt-tests increases Type I error (false positive) rate.

    • Formula: Familywise Error=1−(1−α)n\text{Familywise Error} = 1 - (1 - \alpha)^n

    • Example: For 33 tests at α=0.05\alpha = 0.05: 1−(1−0.05)3=0.14271 - (1 - 0.05)^3 = 0.1427

Concept of ANOVA
  • Omnibus Test: Tests overall null hypothesis that all group means are equal (H0:xˉ1=xˉ2=⋯=xˉkH_0: \bar{x}_1 = \bar{x}_2 = \dots = \bar{x}_k).

    • Statistically significant result = at least one group differs (does not specify which).

  • F-Score Logic:

    • F-score=SignalNoise=Variation ExplainedVariation Unexplained=MSMMSR\text{F-score} = \frac{\text{Signal}}{\text{Noise}} = \frac{\text{Variation Explained}}{\text{Variation Unexplained}} = \frac{MS_M}{MS_R}

  • Formula Breakdown:

    • Model df: dfM=k−1df_M = k - 1 (kk = number of groups)

    • Residual df: dfR=n−kdf_R = n - k (nn = total sample size)

    • Mean Square Model: MSM=SSMdfMMS_M = \frac{SS_M}{df_M}

    • Mean Square Residual: MSR=SSRdfRMS_R = \frac{SS_R}{df_R}

    • F-Statistic: F=MSMMSRF = \frac{MS_M}{MS_R}

Post-Hoc Tests and Reporting
  • Post-Hoc Analysis: Performed after a significant ANOVA (e.g., Tukey HSD) to find which specific groups differ while controlling error.

  • Rule: Only run post-hoc tests if main ANOVA is significant (p<0.05p < 0.05).

  • APA Reporting Format:

    • Significant: Report degrees of freedom, FF-score, and pp-value (e.g., F(3,296)=149.30F(3, 296) = 149.30, p<0.001p < 0.001).

    • Not Significant: Report degrees of freedom, FF-score, and exact pp-value (e.g., F(3,296)=0.57F(3, 296) = 0.57, p=0.636p = 0.636). Include leading zero if F<1F < 1.