Comparing More Than Two Conditions: One-Way Between-Subjects and Repeated-Measures ANOVA and Non-Parametric Tests

Methodological Overview: Comparing Experimental Conditions

• Statistical analysis often requires determining whether differences exist across distinct experimental treatments, intervention phases, or observational groups.

• The foundational decision tree for selecting an inferential test depends on three core parameters:

  • Number of conditions or groups being evaluated (two vs. more than two).

  • Experimental design / participant allocation (between-subjects vs. within-subjects).

  • Distributional properties and measurement level of the dependent variable (scale data meeting parametric assumptions vs. ordinal data or non-normally distributed scale data).

Foundational Framework: Comparing Two Conditions

• Between-Subjects Designs (Different Groups / Independent Samples):

  • Each individual participant is exposed to only one level of the independent variable.

  • Data organization in software packages (e.g., JASP) follows a long format requiring a dedicated grouping variable column (e.g., group: "Picture" vs. "Real spider") alongside the dependent variable score column (e.g., anxiety).

  • Parametric Option: Independent Samples $t$-Test:

  • Requires continuous (interval or ratio) scale data that are normally distributed within each condition.

  • Requires testing the assumption of homogeneity of variance using Levene's test of equality of variances.

  • If Levene's test is non-significant ($p > .05$), equal variances are assumed, and the standard Student's $t$-test is reported.

  • If Levene's test is significant ($p < .05$), equal variances are violated, and Welch's $t$-test correction must be utilized.

  • Non-Parametric Option: Mann-Whitney $U$-Test:

  • Applied when dependent data are ordinal, or when scale data deviate substantially from normality (as indicated by a statistically significant Shapiro-Wilk test, where $p < .05$).

• Within-Subjects Designs (Same Group / Paired Samples / Repeated Measures):

  • The same participants provide data under both experimental conditions.

  • Data organization in software packages follows a wide format where each participant represents a single row, and different conditions occupy separate columns (e.g., control and experimental).

  • Parametric Option: Paired Samples $t$-Test:

  • Requires continuous scale data with normally distributed difference scores.

  • Non-Parametric Option: Wilcoxon Signed-Ranks Test:

  • Applied when paired difference scores are ordinal or violate normality assumptions.

• Inferential Decision Thresholds:

  • If the test probability is greater than $.05$ ($p > .05$): The result is not statistically significant; the null hypothesis ($H_0$) is retained; there is no evidence of a difference between conditions.

  • If the test probability is less than $.05$ ($p < .05$): The result is statistically significant; the null hypothesis ($H_0$) is rejected; a statistically significant difference between conditions is confirmed.

Transition to Multiple Conditions and the Multiple Comparisons Problem

• In research involving three or more groups or measurement time points (e.g., assessing the comparative efficacy of three active anti-depressant pharmacotherapies—Drug A, Drug B, and Drug C—against a Placebo control using Beck Depression Inventory [BDI] scores):

  • Conducting multiple pairwise independent $t$-tests is unacceptable practice for two critical methodological reasons:

  • Absence of Omnibus Assessment: Pairwise $t$-tests cannot evaluate whether an overall, systemic difference exists across the entire condition set simultaneously.

  • Inflation of Type I Error (False Positive Rate):

    • Running sequential $t$-tests drastically inflates the familywise (experimentwise) error rate—the aggregate probability of making at least one Type I error across all comparisons.

    • For four experimental groups, evaluating all pairs requires $\binom{4}{2} = 6$ separate pairwise tests.

    • The familywise error rate ($\alpha_{\text{family}}$) rises according to the formula:         $\alpha_{\text{family}} = 1 - (1 - \alpha)^c$         where $c$ is the number of pairwise comparisons and $\alpha = 0.05$.

    • Across $6$ pairwise comparisons, the true Type I error rate escalates to approximately $26\%$ ($1 - (1 - 0.05)^6 = 1 - 0.735 = 0.265$), creating an unacceptably high chance of false-positive discovery.

  • Analysis of Variance (ANOVA) resolves this issue by executing a single omnibus test that maintains the overarching false-positive risk at the designated $\alpha = .05$ threshold.

Core Principles of Analysis of Variance (ANOVA)

• Terminology and Definitions:

  • ANOVA (Analysis of Variance): An inferential statistical procedure used to test for differences between two or more population means by partitioning total variability.

  • Factor: The categorical independent variable (e.g., "Drug Type" or "Hair Color").

  • Levels: The specific conditions or subgroups within a factor (e.g., Drug A, Drug B, Drug C, Placebo).

• Hypotheses in ANOVA:

  • Null Hypothesis ($H_0$): The omnibus null states that all population condition means are equal:     $H_0: \mu_1 = \mu_2 = \dots = \mu_k$

  • Alternative Hypothesis ($H_1$): At least one population condition mean differs significantly from the others:     $H_1: \text{At least one } \mu_i \neq \mu_j$

• Partitioning of Variability:

  • Total variance within an experimental dataset is decomposed into two distinct components:

  • Between-Groups Variability: Reflects variability between group means, driven by:

    • Systematic treatment effects (the impact of the experimental manipulation).

    • Individual differences (biological, cognitive, or temperamental variability among participants).

    • Experimental error (uncontrolled environmental fluctuations or measurement noise).

  • Within-Groups Variability (Error Variance):

    • Reflects variability among scores within the same treatment condition.

    • Because all participants within a single group receive the identical treatment, this variability is solely driven by:

    • Individual differences.

    • Experimental error.

• The $F$-Statistic Ratio:

  • The test statistic $F$ is defined mathematically as the ratio of between-groups variance to within-groups variance:     $F = \frac{\text{Variance between groups}}{\text{Variance within groups}} = \frac{\text{Treatment Effect} + \text{Individual Differences} + \text{Experimental Error}}{\text{Individual Differences} + \text{Experimental Error}}$

  • Behavior Under the Null Hypothesis ($H_0$):

  • If $H_0$ is true, there is zero treatment effect ($\text{Treatment Effect} = 0$).

  • The numerator reduces to individual differences and experimental error, matching the denominator.

  • Consequently, the expected value of $F$ under $H_0$ is approximately $1.0$ ($F \approx 1$).

  • Behavior Under the Alternative Hypothesis ($H_1$):

  • If a true treatment effect exists ($\text{Treatment Effect} > 0$), the numerator exceeds the denominator.

  • Consequently, the resulting $F$-statistic is substantially greater than $1.0$ ($F > 1$).

The $F$-Distribution, Degrees of Freedom, and Reporting Conventions

• Properties of the $F$-Distribution:

  • The $F$-distribution is positively skewed and bounded at zero ($F \ge 0$).

  • Its precise mathematical shape is governed by two separate degrees of freedom ($df$):

  • Between-groups degrees of freedom ($df_{\text{between}}$):       $df_{\text{between}} = k - 1$       where $k$ is the total number of conditions or levels.

  • Within-groups / error degrees of freedom ($df_{\text{within}}$):       $df_{\text{within}} = N - k$       where $N$ is the total sample size across all conditions.

  • Total degrees of freedom ($df_{\text{total}}$):       $df_{\text{total}} = N - 1 = df_{\text{between}} + df_{\text{within}}$

• Follow-Up (Post-Hoc) Comparisons:

  • Because the omnibus $F$-test assesses only whether at least one difference exists across the entire condition array, a significant $F$-test ($p < .05$) does not indicate which specific pairs of means differ.

  • Post-hoc tests execute pairwise comparisons between condition means while strictly adjusting the familywise error rate.

  • Standard post-hoc adjustments include Tukey's Honestly Significant Difference (HSD), Scheffé, and Bonferroni corrections.

• Reporting Standards:

  • General APA reporting syntax: $F(df_{\text{between}}, df_{\text{within}}) = F\text{-value}, p = p\text{-value}$.

  • Standard reporting includes measures of effect size and $95\%$ confidence intervals ($95\%\text{ CI}$).

• Effect Size Formulations:

  • Partial Eta Squared ($\eta_p^2$):

  • Quantifies the proportion of total variance in the dependent variable explained by the independent variable factor:       $\eta_p^2 = \frac{SS_{\text{effect}}}{SS_{\text{effect}} + SS_{\text{error}}}$

  • Omega Squared ($\omega^2$):

  • An unbiased population effect size estimator that adjusts for sample size and degrees of freedom, highly recommended when handling smaller sample sizes:       $\omega^2 = \frac{SS_{\text{effect}} - (df_{\text{effect}} \times MS_{\text{error}})}{SS_{\text{total}} + MS_{\text{error}}}$

One-Way Between-Subjects ANOVA

• Definition and Scope:

  • Tests whether statistically significant differences exist across three or more independent group means derived from mutually exclusive participant samples.

• Parametric Assumptions:

  • Continuous scale (interval or ratio) dependent data.

  • Normal distribution of residuals or scores within each level of the factor (verified via Shapiro-Wilk testing or standardized residual Quantile-Quantile [Q-Q] plots).

  • Mutual independence of observations (each participant provides one score; participants do not influence one another).

  • Homogeneity of error variance (verified via Levene's test of equality of variances).

  • If Levene's test is non-significant ($p > .05$), equal variances are assumed.

  • If Levene's test is significant ($p < .05$), the assumption is violated; a robust alternative must be used: Welch's ANOVA or Brown-Forsythe correction.

• Non-Parametric Counterpart:

  • The Kruskal-Wallis test is used when between-subjects data violate normality or are measured on an ordinal scale.

Case Study 1: Pain Thresholds and Hair Color (McClave & Dietrich, 1991)

• Experimental Overview:

  • Independent Variable: Hair color category with $4$ levels:

  • Light Blond

  • Dark Blond

  • Light Brunette

  • Dark Brunette

  • Dependent Variable: Quantitative pain threshold tolerance score.

  • Hypotheses:

  • $H_0$: There is no difference in mean pain threshold across hair color groups ($\mu_{\text{Dark Blond}} = \mu_{\text{Dark Brunette}} = \mu_{\text{Light Blond}} = \mu_{\text{Light Brunette}}$).

  • $H_1$: At least one hair color group exhibits a different mean pain threshold.

• Descriptive Statistics:

  • Dark Blond: Mean = $51.200$, $SD = 9.284$, $N = 5$.

  • Dark Brunette: Mean = $37.400$, $SD = 8.325$, $N = 5$.

  • Light Blond: Mean = $59.200$, $SD = 8.526$, $N = 5$.

  • Light Brunette: Mean = $42.500$, $SD = 5.447$, $N = 4$.

  • Total sample size: $N = 19$.

• Assumption Testing:

  • Levene's Test for Equality of Variances: $F(3, 15) = 0.493$, $p = 0.692$.

  • Because $p > .05$, homogeneity of variance is maintained; no correction is required.

  • Standardized residuals plotted against theoretical quantiles in a Q-Q plot adhere closely to the linear diagonal, confirming residual normality.

• Omnibus ANOVA Results (Type III Sum of Squares):

  • Between-groups (Hair Color): Sum of Squares ($SS$) = $1360.726$, $df = 3$, Mean Square ($MS$) = $453.575$, $F = 6.791$, $p = 0.004$, $\eta_p^2 = 0.576$, $\omega^2 = 0.478$.

  • Within-groups (Residuals): $SS = 1001.800$, $df = 15$, $MS = 66.787$.

  • Result interpretation: A statistically significant difference exists in mean pain tolerance across hair colors, $F(3, 15) = 6.791$, $p = .004$, $\omega^2 = 0.478$.

• Post-Hoc Tukey HSD Pairwise Comparisons (Adjusted for Family of 4):

  • Dark Blond vs. Dark Brunette: Mean Difference = $13.800$, $SE = 5.169$, $t = 2.670$, $p_{\text{tukey}} = 0.074$.

  • Dark Blond vs. Light Blond: Mean Difference = $-8.000$, $SE = 5.169$, $t = -1.548$, $p_{\text{tukey}} = 0.436$.

  • Dark Blond vs. Light Brunette: Mean Difference = $8.700$, $SE = 5.482$, $t = 1.587$, $p_{\text{tukey}} = 0.415$.

  • Dark Brunette vs. Light Blond: Mean Difference = $-21.800$, $SE = 5.169$, $t = -4.218$, $p_{\text{tukey}} = 0.004$ (statistically significant, $p < .01$).

  • Dark Brunette vs. Light Brunette: Mean Difference = $-5.100$, $SE = 5.482$, $t = -0.930$, $p_{\text{tukey}} = 0.789$.

  • Light Blond vs. Light Brunette: Mean Difference = $16.700$, $SE = 5.482$, $t = 3.046$, $p_{\text{tukey}} = 0.037$ (statistically significant, $p < .05$).

• Conclusion:

  • Participants with Light Blond hair demonstrated significantly higher pain thresholds than participants with Dark Brunette hair ($p = .004$) and Light Brunette hair ($p = .037$).

  • All remaining pairwise comparisons did not achieve statistical significance ($p > .05$).

Case Study 2: Social Attractiveness and Facebook Friends (Moore et al., 2012)

• Experimental Overview:

  • Research Question: Does the total number of Facebook friends displayed on an individual's social networking profile influence perceived social attractiveness?

  • Independent Variable: Number of friends across $5$ between-subjects profile conditions:

  • $102$ friends

  • $302$ friends

  • $502$ friends

  • $702$ friends

  • $902$ friends

  • Dependent Variable: Standardized rating score of social attractiveness.

  • Hypotheses:

  • $H_0$: Mean perceived social attractiveness is identical across friend counts.

  • $H_1$: Mean perceived social attractiveness differs significantly across friend counts.

• Descriptive Statistics:

  • Condition $102$ friends: Mean = $3.817$, $SD = 0.999$, $N = 24$.

  • Condition $302$ friends: Mean = $4.879$, $SD = 0.851$, $N = 33$.

  • Condition $502$ friends: Mean = $4.562$, $SD = 1.070$, $N = 26$.

  • Condition $702$ friends: Mean = $4.407$, $SD = 1.428$, $N = 30$.

  • Condition $902$ friends: Mean = $3.990$, $SD = 1.023$, $N = 21$.

  • Total sample size: $N = 134$.

• Assumption Testing and Violation Handling:

  • Levene's Test for Equality of Variances: $F(4, 129) = 2.607$, $p = 0.039$.

  • Because $p < .05$, the assumption of equal variances is violated.

  • Standard Fisher ANOVA cannot be reported without correction; robust alternatives must be interpreted:

  • Brown-Forsythe Correction: $SS = 19.890$, $df_1 = 4.000$, $df_2 = 114.185$, $MS = 4.973$, $F = 4.184$, $p = 0.003$, $\eta_p^2 = 0.114$, $\omega^2 = 0.086$.

  • Welch's Correction: $SS = 19.890$, $df_1 = 4.000$, $df_2 = 61.144$, $MS = 4.973$, $F = 5.445$, $p < 0.001$, $\eta_p^2 = 0.114$, $\omega^2 = 0.086$.

  • Reporting standard: There was a significant difference in mean rated social attractiveness between Facebook profiles, Welch's $F(4.00, 61.14) = 5.445$, $p < .001$, $\eta_p^2 = 0.114$.

• Parametric Post-Hoc Comparisons (Tukey Correction, Family of 5):

  • $102$ vs. $302$: Mean Difference = $-1.062$, $SE = 0.294$, $t = -3.613$, $p_{\text{tukey}} = 0.004$ (significant, $p < .01$).

  • $102$ vs. $502$: Mean Difference = $-0.745$, $SE = 0.310$, $t = -2.402$, $p_{\text{tukey}} = 0.121$.

  • $102$ vs. $702$: Mean Difference = $-0.590$, $SE = 0.300$, $t = -1.966$, $p_{\text{tukey}} = 0.288$.

  • $102$ vs. $902$: Mean Difference = $-0.174$, $SE = 0.327$, $t = -0.531$, $p_{\text{tukey}} = 0.984$.

  • $302$ vs. $502$: Mean Difference = $0.317$, $SE = 0.287$, $t = 1.104$, $p_{\text{tukey}} = 0.804$.

  • $302$ vs. $702$: Mean Difference = $0.472$, $SE = 0.276$, $t = 1.708$, $p_{\text{tukey}} = 0.433$.

  • $302$ vs. $902$: Mean Difference = $0.888$, $SE = 0.306$, $t = 2.904$, $p_{\text{tukey}} = 0.035$ (significant, $p < .05$).

  • $502$ vs. $702$: Mean Difference = $0.155$, $SE = 0.294$, $t = 0.528$, $p_{\text{tukey}} = 0.984$.

  • $502$ vs. $902$: Mean Difference = $0.571$, $SE = 0.321$, $t = 1.776$, $p_{\text{tukey}} = 0.392$.

  • $702$ vs. $902$: Mean Difference = $0.416$, $SE = 0.312$, $t = 1.335$, $p_{\text{tukey}} = 0.670$.

• Non-Parametric Execution: Kruskal-Wallis Test:

  • Used as an alternative or robustness test against variance heterogeneity.

  • Kruskal-Wallis Test: Factor = Friends, Statistic ($H$) = $17.053$, $df = 4$, $p = 0.002$.

  • Dunn's Post-Hoc Pairwise Comparisons:

  • $102 - 302$: $z = -3.720$, $W_i = 46.250$, $W_j = 84.909$, unadjusted $p < .001$, $p_{\text{bonf}} < .001$, $p_{\text{holm}} < .001$.

  • $102 - 502$: $z = -2.373$, $W_i = 46.250$, $W_j = 72.269$, unadjusted $p = 0.009$, $p_{\text{bonf}} = 0.088$, $p_{\text{holm}} = 0.071$.

  • $102 - 702$: $z = -2.305$, $W_i = 46.250$, $W_j = 70.700$, unadjusted $p = 0.011$, $p_{\text{bonf}} = 0.106$, $p_{\text{holm}} = 0.074$.

  • $102 - 902$: $z = -0.665$, $W_i = 46.250$, $W_j = 53.952$, unadjusted $p = 0.253$, $p_{\text{bonf}} = 1.000$, $p_{\text{holm}} = 0.506$.

  • $302 - 502$: $z = 1.244$, $W_i = 84.909$, $W_j = 72.269$, unadjusted $p = 0.107$, $p_{\text{bonf}} = 1.000$, $p_{\text{holm}} = 0.322$.

  • $302 - 702$: $z = 1.454$, $W_i = 84.909$, $W_j = 70.700$, unadjusted $p = 0.073$, $p_{\text{bonf}} = 0.730$, $p_{\text{holm}} = 0.322$.

  • $302 - 902$: $z = 2.863$, $W_i = 84.909$, $W_j = 53.952$, unadjusted $p = 0.002$, $p_{\text{bonf}} = 0.021$, $p_{\text{holm}} = 0.019$.

  • $502 - 702$: $z = 0.151$, $W_i = 72.269$, $W_j = 70.700$, unadjusted $p = 0.440$, $p_{\text{bonf}} = 1.000$, $p_{\text{holm}} = 0.506$.

  • $502 - 902$: $z = 1.612$, $W_i = 72.269$, $W_j = 53.952$, unadjusted $p = 0.054$, $p_{\text{bonf}} = 0.535$, $p_{\text{holm}} = 0.321$.

  • $702 - 902$: $z = 1.519$, $W_i = 70.700$, $W_j = 53.952$, unadjusted $p = 0.064$, $p_{\text{bonf}} = 0.643$, $p_{\text{holm}} = 0.322$.

• Theoretical Implications:

  • The relationship between friend count and social attractiveness exhibits an inverted-U shape: perceived social attractiveness peaks around $302$ friends and declines significantly at both deficient ($102$) and excessively high ($902$) friend levels.

One-Way Repeated Measures ANOVA

• Definition and Scope:

  • Tests whether statistically significant differences exist across three or more condition means gathered from the exact same participant group.

• Enhanced Statistical Power:

  • In repeated measures designs, every participant acts as their own control across conditions.

  • Individual differences (the primary driver of within-groups error in independent designs) are mathematically isolated and removed from the denominator error term:     $F = \frac{\text{Treatment Effect} + \text{Experimental Error}}{\text{Experimental Error}}$

  • By eliminating individual difference variance from the error baseline, the denominator shrinks substantially, resulting in a more sensitive, statistically powerful test for detecting genuine treatment effects.

• The Assumption of Sphericity:

  • Definition: Sphericity requires that the population variances of the differences between all possible combinations of within-subjects condition pairs are equal:     $\text{Var}(X_1 - X_2) = \text{Var}(X_1 - X_3) = \text{Var}(X_2 - X_3)$

  • Statistical Assessment: Mauchly's test of sphericity evaluates the null hypothesis that condition difference variances are equal.

  • If Mauchly's test is non-significant ($p \ge .05$): Sphericity is assumed; standard uncorrected ANOVA output is interpreted.

  • If Mauchly's test is significant ($p < .05$): Sphericity is violated, inflating Type I error rates. The degrees of freedom must be multiplied by an epsilon factor ($\varepsilon \le 1.0$) to adjust the critical $F$-threshold:

    • If Greenhouse-Geisser $\varepsilon < 0.75$: Apply the conservative Greenhouse-Geisser correction.

    • If Greenhouse-Geisser $\varepsilon > 0.75$: Apply the less conservative Huynh-Feldt correction.

• Non-Parametric Counterpart:

  • The Friedman test is the non-parametric alternative for repeated measures designs involving ordinal data or severely skewed continuous data.

Case Study 3: Bush Tucker Trials (Field, 2017)

• Experimental Overview:

  • Design: Within-subjects repeated measures ($N = 8$ celebrities).

  • Independent Variable: Four distinct bush tucker trial food conditions:

  • Stick insect

  • Kangaroo testicle

  • Fish eye

  • Witchetty grub

  • Dependent Variable: Latency in seconds of consuming the food item prior to retching.

• Participant Raw Data Matrix and Means:

  • Celebrity 1: Stick insect = $8$, Kangaroo testicle = $7$, Fish eye = $1$, Witchetty grub = $6$, Participant Mean = $5.50$.

  • Celebrity 2: Stick insect = $9$, Kangaroo testicle = $5$, Fish eye = $2$, Witchetty grub = $5$, Participant Mean = $5.25$.

  • Celebrity 3: Stick insect = $6$, Kangaroo testicle = $2$, Fish eye = $3$, Witchetty grub = $8$, Participant Mean = $4.75$.

  • Celebrity 4: Stick insect = $5$, Kangaroo testicle = $3$, Fish eye = $1$, Witchetty grub = $9$, Participant Mean = $4.50$.

  • Celebrity 5: Stick insect = $8$, Kangaroo testicle = $4$, Fish eye = $5$, Witchetty grub = $8$, Participant Mean = $6.25$.

  • Celebrity 6: Stick insect = $7$, Kangaroo testicle = $5$, Fish eye = $6$, Witchetty grub = $7$0, Participant Mean = $6.25$.

  • Celebrity 7: Stick insect = $10$, Kangaroo testicle = $2$, Fish eye = $7$, Witchetty grub = $2$, Participant Mean = $5.25$.

  • Celebrity 8: Stick insect = $12$, Kangaroo testicle = $6$, Fish eye = $8$, Witchetty grub = $1$, Participant Mean = $6.75$.

  • Condition Means:

  • Stick insect: $M = 8.125$

  • Kangaroo testicle: $M = 4.250$

  • Fish eye: $M = 4.125$

  • Witchetty grub: $M = 4.750$

• Assumption Testing for Sphericity:

  • Mauchly's Test of Sphericity: Mauchly's $W = 0.136$, Approximate $\chi^2 = 11.406$, $df_{\text{sphericity}} = 5$, $p = 0.047$.

  • Epsilon estimates: Greenhouse-Geisser $\varepsilon = 0.533$, Huynh-Feldt $\varepsilon = 0.666$, Lower Bound $\varepsilon = 0.333$.

  • Assessment: Because Mauchly's $p = .047 < .05$, sphericity is violated. Because Greenhouse-Geisser $\varepsilon = 0.533 < 0.75$, the Greenhouse-Geisser correction is mandated.

• Within-Subjects ANOVA Results:

  • Uncorrected Output (Incorrect due to violated sphericity):

  • Bush tucker effect: $SS = 83.125$, $df = 3.000$, $MS = 27.708$, $F = 3.794$, $p = 0.026$, $\eta_p^2 = 0.351$, $\omega^2 = 0.239$.

  • Residuals: $SS = 153.375$, $df = 21.000$, $MS = 7.304$.

  • Greenhouse-Geisser Corrected Output (Correct):

  • Bush tucker effect: $SS = 83.125$, $df = 1.599$, $MS = 52.001$, $F = 3.794$, $p = 0.063$, $\eta_p^2 = 0.351$, $\omega^2 = 0.239$.

  • Residuals: $SS = 153.375$, $df = 11.190$, $MS = 13.707$.

• Conclusion and Reporting:

  • Formal statement: There was no significant difference in how repelling bush tucker trial foods were to celebrities, Greenhouse-Geisser corrected $F(1.60, 11.19) = 3.79$, $p = .063$, $\eta_p^2 = 0.351$.

  • Sphericity correction alters the statistical outcome from significant ($p = .026$) to non-significant ($p = .063$), demonstrating the importance of correcting degrees of freedom to avoid false-positive conclusions. No post-hoc tests are conducted.

Case Study 4: Response-Cost Technique Over Time (Gravetter & Wallnau, 1996)

• Experimental Overview:

  • Study objective: Assess the sustained efficacy of a response-cost behavior modification intervention designed to reduce emotional outbursts in children.

  • Design: Longitudinal one-way within-subjects repeated measures.

  • Measurement Intervals ($4$ levels of Time):

  • Baseline (Before treatment)

  • One week post-treatment

  • One month post-treatment

  • Six months post-treatment

  • Dependent Variable: Total count of behavioral outbursts.

• Repeated Measures ANOVA Results:

  • Omnibus Test (Type III Sum of Squares):

  • Time effect: $SS = 77.000$, $df = 3$, $MS = 25.667$, $F = 21.000$, $p < 0.001$, $\eta_p^2 = 0.875$, $\omega^2 = 0.696$.

  • Residuals: $SS = 11.000$, $df = 9$, $MS = 1.222$.

  • Reporting statement: There was a statistically significant difference in the mean number of behavioral outbursts observed over time, $F(3, 9) = 21.00$, $p < .001$, $\omega^2 = 0.696$.

• Post-Hoc Pairwise Comparisons (Bonferroni Corrected, Family of 6 Comparisons):

  • Before vs. One week: Mean Difference = $4.500$, $SE = 0.645$, $t = 6.971$, $p_{\text{bonf}} = 0.036$ (statistically significant, $p < .05$).

  • Before vs. One month: Mean Difference = $5.000$, $SE = 0.913$, $t = 5.477$, $p_{\text{bonf}} = 0.072$.

  • Before vs. Six months: Mean Difference = $5.500$, $SE = 0.866$, $t = 6.351$, $p_{\text{bonf}} = 0.047$ (statistically significant, $p < .05$).

  • One week vs. One month: Mean Difference = $0.500$, $SE = 0.645$, $t = 0.775$, $p_{\text{bonf}} = 1.000$.

  • One week vs. Six months: Mean Difference = $1.000$, $SE = 0.408$, $t = 2.449$, $p_{\text{bonf}} = 0.550$.

  • One month vs. Six months: Mean Difference = $0.500$, $SE = 1.041$, $t = 0.480$, $p_{\text{bonf}} = 1.000$.

• Non-Parametric Execution: Friedman Test:

  • Used as an alternative when outburst counts violate continuous scale or normality assumptions.

  • Friedman Test: Factor = Time, Chi-Squared ($\chi^2$) = $9.081$, $df = 3$, $p = 0.028$, Kendall's $W = 0.521$.

  • Conover's Post-Hoc Pairwise Comparisons (Grouped by Subject, $df = 9$):

  • Before vs. One week: $t = 1.657$, $W_i = 16.000$, $W_j = 10.000$, unadjusted $p = 0.132$, $p_{\text{bonf}} = 0.791$, $p_{\text{holm}} = 0.528$.

  • Before vs. One month: $t = 2.209$, $W_i = 16.000$, $W_j = 8.000$, unadjusted $p = 0.055$, $p_{\text{bonf}} = 0.327$, $p_{\text{holm}} = 0.273$.

  • Before vs. Six months: $t = 2.762$, $W_i = 16.000$, $W_j = 6.000$, unadjusted $p = 0.022$, $p_{\text{bonf}} = 0.132$, $p_{\text{holm}} = 0.132$.

  • One week vs. One month: $t = 0.552$, $W_i = 10.000$, $W_j = 8.000$, unadjusted $p = 0.594$, $p_{\text{bonf}} = 1.000$, $p_{\text{holm}} = 1.000$.

  • One week vs. Six months: $t = 1.105$, $W_i = 10.000$, $W_j = 6.000$, unadjusted $p = 0.298$, $p_{\text{bonf}} = 1.000$, $p_{\text{holm}} = 0.894$.

  • One month vs. Six months: $t = 0.552$, $W_i = 8.000$, $W_j = 6.000$, unadjusted $p = 0.594$, $p_{\text{bonf}} = 1.000$, $p_{\text{holm}} = 1.000$.

• Clinical and Theoretical Conclusion:

  • The parametric analysis confirms that the response-cost intervention produces a statistically significant, lasting decline in emotional outbursts from baseline to one week ($p = .036$) and from baseline to six months ($p = .047$).

  • The lack of significant difference between one week, one month, and six months demonstrates that behavioral improvements remain stable across long-term follow-up.

Comprehensive Statistical Decision Framework

• Condition Taxonomy for Testing Differences Across More Than Two Conditions:

  • Independent Samples (Between-Subjects Designs):

  • Scale data, normally distributed, equal variances: One-Way Between-Subjects ANOVA (Standard Fisher's $F$-test).

  • Scale data, normally distributed, unequal variances (Levene's $p < .05$): Welch's ANOVA or Brown-Forsythe corrected ANOVA.

  • Ordinal data OR scale data violating normality: Kruskal-Wallis test (followed by Dunn's post-hoc comparisons with Bonferroni or Holm corrections).

  • Related Samples (Within-Subjects / Repeated Measures Designs):

  • Scale data, normally distributed, sphericity met (Mauchly's $p \ge .05$): One-Way Repeated Measures ANOVA (Sphericity Assumed).

  • Scale data, normally distributed, sphericity violated (Mauchly's $p < .05$):

    • If Greenhouse-Geisser $\varepsilon < 0.75$: Repeated Measures ANOVA with Greenhouse-Geisser correction.

    • If Greenhouse-Geisser $\varepsilon > 0.75$: Repeated Measures ANOVA with Huynh-Feldt correction.

  • Ordinal data OR scale data violating normality: Friedman test (followed by Conover's post-hoc comparisons).

Foundational Literature & Study Resources

• Primary Statistical Texts:

  • Coolican, H. Research Methods and Statistics in Psychology, Chapter 20 (Analysis of Variance).

  • Wilson, S., & MacLean, R. Research Methods and Data Analysis for Psychology, Chapter 17 (Comparing More Than Two Means).

• JASP Computational Software Manuals:

  • Goss-Sampson, M. A. Statistical Analysis in JASP: A Guide for Students, pp. 84–101 (ANOVA and Repeated Measures).

  • Navarro, D. J., Foxcroft, D. R., & Faulkenberry, T. J. Learning Statistics with JASP: A Tutorial for Psychology Students and Other Beginners, Chapter 12 (Comparing Several Means).