One-Way ANOVA Notes

One-Way ANOVA with 3 Groups

Introduction

• Lecture 4, Part 1 of PSYC 305: Statistics for Experimental Design.
• Date: 2025-05-07
• Instructor: Dr. Jens Kreitewolf

Computational Formulas

• Formulas for Sum of Squares (SS), Variance, and Standard Deviation (SD) are essential for calculations in ANOVA.

Example: One-Way ANOVA

• Scenario: A researcher investigates if additional readings increase interest in statistics.
• Method: Participants are randomly assigned to one of three conditions:
  • Stat1 & Stat2: Reading one of two statistics books intended for the general public.
  • NoStat: Control condition with no statistics-related reading material.

Experimental Design

• A random sample of students is taken, and each student is randomly assigned to read one book.
• Interest ratings are recorded for each student.
• Data:
  • Stat1: 82, 83, 97, 93, 55, 67, 53
  • Stat2: 83, 78, 68, 61, 77, 54, 69, 51, 63
  • NoStat: 38, 59, 55, 66, 45, 52, 52, 61

Variables

• Factor (IV): Book
  • Three levels: Stat1, Stat2, NoStat
• DV: Self-reported interest in taking more statistics courses (0-100).
• Hypotheses:
  • Null Hypothesis ($H<em>0$): $μ</em>1 = μ<em>2 = μ</em>3$ (All means are equal).
  • Alternative Hypothesis ($H_a$): Not all $μ$’s are the same.

ANOVA Results

• Source of Variance Table:
  • Model (Between): SS = 1901.516, df = 2, MS = 950.758, F = 5.896
  • Residual (Within): SS = 3386.317, df = 21, MS = 161.253
  • Total: SS = 5287.833, df = 23, MS = 229.906
• The critical value for $\alpha$ = .05 is 3.47 with $df<em>M$ = 2 and $df</em>R$ = 21.
• The null hypothesis may be rejected because the obtained F ratio (5.9) is greater than 3.47.
• Conclusion: There is a significant difference among the three books in their effect on student interest in statistics, $F(2, 21) = 5.9$, $p < .05$, with a large effect size, $\omega^2 = .29$.

APA Style Summary

• To investigate whether reading one of three different books had an effect on interest in taking statistics courses (0-100, with higher scores meaning more interest), we conducted a one-way between-subjects ANOVA.
• Shorthand notation: The Signal and the Noise (Stat1), Statistical Rethinking (Stat2), and Blink (NoStat) as the three book conditions.
• This analysis revealed a significant effect of book on interest, $F(2, 21) = 5.9$, $p < .05$, $\omega^2 = .29$.
• Post hoc tests are needed to understand which pairs of means differ significantly.

Post Hoc Tests

• When $H_0$ is rejected in ANOVA, we infer that at least one pair of means is different.
• However, we may not know which pairs of means are different (e.g., $μ<em>1 ≠ μ</em>2 ≠ μ<em>3$, $μ</em>1 = μ<em>2 ≠ μ</em>3$, $μ<em>1 ≠ μ</em>2 = μ_3$).

Omnibus Test

• The $F$ ratio or $F$ test gives a global effect of the independent variable on the dependent variable (omnibus or overall test).
• It does not tell us which pairs of means are different.
• We need to perform post hoc tests to make further inferences about which means are different.
• Post hoc (a posteriori/unplanned) comparisons are decided upon after the experiment.
• In the case of ANOVA, used if three or more means were compared.

Post Hoc Comparisons

• Two post-hoc tests:
  1. Scheffé
  2. Tukey’s Honestly Significant Difference (HSD) test

Scheffé’s Test

• Can be used if groups have different sample sizes.
• Less sensitive to departures from the assumption of normality and equal variances in the population.
• It is the most conservative test (very unlikely to reject $H_0$).
• Good choice if you wish to avoid Type I errors, but it has lower power to detect differences.
• But, it uses a larger critical value.
• The critical value for this test is determined as follows:
  • Obtain the critical value of $F$ with $df<em>1 = k − 1$ and $df</em>2 = N − k$ (in other words, obtain the critical value as usual).
  • Multiply this value by $k − 1$.
• Uses $F$ ratio to test for a significant difference between any two means (e.g., $H<em>0: μ</em>1 = μ_2$):
• The $SS$ for the specific comparisons need to be calculated based on which groups are being compared and the residual $SS$ is simply the $SS_{residual}$ from the main analysis.

Steps in Scheffé’s Test

• Calculate the $SS<em>{comparison}$, $MS</em>{comparison}$, and $F_{comparison}$.
• Compare the observed $F<em>{comparison}$ to $(k − 1)(F</em>{critical(k − 1), (N − k)})$.
• If the observed $F_{comparison}$ is greater, conclude that the pair of means is significantly different from 0.
• If not, the pair of means being compared is not significantly different.

Scheffé’s Test: Example

• From the previous ANOVA results:
  • The critical value for $\alpha$ = .05 is 3.47 with $df<em>M$ = 2 and $df</em>R$ = 21
  • The critical value for Scheffé’s Test is $(3 − 1) \times 3.47 = 6.94$
• We need to calculate the SS, MS and observed F for each pairwise mean comparison.
• The groups being compared receive coefficients (c) of -1 and 1; the group(s) excluded from this particular comparison receive a coefficient of 0.
• All $SS_{comparison}$ are calculated using means of each level of the independent variable
  • $\phi = c<em>1 X</em>{bar1} + c<em>2 X</em>{bar2} + c<em>3 X</em>{bar3}$
  • $SS<em>{comparison} = \frac{\phi^2}{\sum{(c</em>i^2/n_i)}}$

Scheffé’s Test: Calculation

• Scheffé’s Test consists of a comparison of $k = 2$ group means, so $df = k − 1 = 1$
• Therefore $MS<em>{comparison} = \frac{SS</em>{comparison}}{df} = SS_{comparison}$
• $F<em>{comparison} = \frac{MS</em>{comparison}}{MS_{residual}}$
• $MS_{residual} = 161.2$ in this example
• Therefore, $F_{comparison} = \frac{290.555}{161.253} = 1.808$
• Compare to the critical value for Scheffé’s Test, which is $(3 − 1) \times 3.47 = 6.94$
• 1.808 < 6.94, thus the average interest in statistics among students who read Stats1 and students who read Stats2 are not significantly different

Homework

• There are two more pairs of means to compare:
  • Stats1 vs. No Stats: $H<em>0: μ</em>1 = μ_3$
  • Stats2 vs. No Stats: $H<em>0: μ</em>2 = μ_3$

Solution to Homework

• $F_{Stat1 vs NoStat} = \frac{1875.927}{161.253} = 11.633$
• $F_{Stat2 vs NoStat} = \frac{785.004}{161.253} = 4.868$
• For comparison, the critical value for Scheffé’s Test was: $(3 − 1) \times 3.47 = 6.94$

Complete APA Style Summary

• To investigate whether reading one of three different books had an effect on interest in taking statistics courses (0-100, with higher scores meaning more interest), we conducted a one-way between-subjects ANOVA. We use short-hand notation to refer to The Signal and the Noise (Stat1), Statistical Rethinking (Stat2), and Blink (NoStat) as the three book conditions. This analysis revealed a significant effect of book on interest, $F(2, 21) = 5.9, p < .05, \omega^2 = .29$.
• To understand the significant overall test, we conducted Scheffé’s post-hoc comparisons among all pairs of group means (with $\&alpha = .05$). Participants in the Stat1 condition (M = 75.71, SD = 17.63) exhibited significantly more interest than those in the NoStat condition (M = 53.50, SD = 8.96). Stat1 also had more interest than the Stat2 condition (M = 67.11, SD = 10.95), and Stat2 had more interest than the NoStat condition, but neither difference was significant. We conclude that reading an interesting book about statistics (The Signal and the Noise) may increase interest more than reading a general book about psychological findings (Blink).