EBP Lec 14: ANOVA I

Introduction

• Greeting to students and recognition of guest lecturer Rahul.

• Acknowledgment of unusual scheduling, as lecturer usually does not have classes on Tuesday.

• Mention of previous lecture being swapped to accommodate other events (optom audiology).

Lecture Agenda

• Topic for today: Analysis of Variance (ANOVA).

• Review of previous topics:

  • Basic statistics, descriptive statistics.

  • Central limit theorem and foundational inferential statistics.

  • Correlation, regression, and t-tests.

• Focus on:

  • Foundations of ANOVA, when to use it, and how to interpret outputs.

  • Application of one-way ANOVA today and discussing more advanced ANOVA concepts tomorrow.

Understanding One-Way ANOVA

Predictor Variable and Levels

• Definition: A one-way ANOVA involves one predictor (independent) variable with three or more levels (categories).

  • Example: Comparing different dog breeds or chocolate brands.

• Levels: Refers to the categories or groups in an independent variable.

  • Example: Comparing weights across Great Danes, Greyhounds, Golden Retrievers, and Jack Russells.

Transitioning from t-Tests to ANOVA

• T-tests can only compare two groups.

• One-way ANOVA is employed when there are three or more groups.

  • Example:

  • T-test: Comparing two chocolate brands.

  • One-way ANOVA: Comparing weights of multiple brands.

• Dependent variable must be measured on an interval or ratio scale.

Conceptual Framework for ANOVA

Model Explanation

• ANOVA compares variance between groups (between-group variance) against variance within groups (within-group variance).

• Model: Predicted (mean) variance due to the categorical factor (e.g., breed), plus a component of error.

• ANOVA accounts for some variance but not all, recognizing the influence of uncontrollable variables.

Justification for Using ANOVA Instead of Multiple t-Tests

• If multiple t-tests were used (e.g., comparing every pair of dog breeds):

  1. Each t-test at significance threshold ($ ext{alpha} = 0.05$) risks Type I error.

  2. Cumulative Type I error rate rises with increased testing, diminishing overall reliability.

  3. ANOVA addresses this by retaining a single Type I error rate across comparisons (family-wise error rate).

General Linear Model

• General approach of statistics including ANOVA and regression.

• Involves:

  • Dependent variable: Outcome variable (e.g., dog weights).

  • Model: Represents systematic variance accounted for by categorical factors.

  • Error: Residual variance not explained by the model.

Assumptions of ANOVA

1. Dependent variable must be interval or ratio.

2. Categorical independent variables with 3 or more levels.

3. Samples are randomly selected (independence).

4. Normal distribution of the outcome variable within the population.

5. Equal variances across groups (homoscedasticity).

   • Violations are permissible if sample sizes are equal.

Key Components of ANOVA Output

• F-statistic: Ratio of between-group variance to within-group variance.

  • High F-value suggests significant differences between groups.

  • Calculation: $F = \frac{\text{Between-group variance}}{\text{Within-group variance}}$.

• Example: Larger values indicate better explanatory power of the model in contrasting group means.

Creating ANOVA Tables

• Structure includes:

  • Source of variance (factor, error).

  • Degrees of freedom (df) for each component.

  • Sum of squares (ss), mean squares (ms), and resulting F-value and p-value.

  • Report all statistical findings as per APA guidelines.

Utilizing ANOVA in Practical Scenarios

Example - Dog Weight Analysis

• Predictive model assumes dog breeds predict variations in weights.

• ANOVA reveals group differences: Great Danes heavier and Jack Russells lighter.

  • Output interpretation: Significant differences affirmed by examining F-statistics and p-values along with Tukey's HSD for post hoc analysis.

Example - Additional Considerations

• Post hoc tests (e.g., Tukey’s test) follow significant ANOVA results to ascertain which specific means differ.

  • Example: Identify specific pair differences in test frequencies across groups.

Summary of Important Points

• A one-way ANOVA is utilized for comparing group means when independent variable comprises three or more categorical levels.

• The dependent variable must be of high data quality (interval or ratio).

• Use ANOVA to avoid compounding of Type I error rates found in multiple t-tests.

• The F-value and p-value in the output guide interpretations for group mean differences.

• Successful reporting includes model justification, paired comparisons, and variance explained (e.g., R-squared) for effective conclusions.

Conclusion

• Continuous engagement and clarifications through examples, including exercises and questions with various outputs.

• Importance of understanding practical applications of statistical insights and methodology in real-world evaluations, particularly in research fields relevant to the course.

Concept of Levels and Predictors

Independent Variable (Predictor)

• Categorical, with multiple levels.

  • 2 levels → use a t-test

  • ≥ 3 levels → need ANOVA

Dependent Variable (Outcome)

• Interval or ratio scale (continuous, quantitative measure).

Example progression:

3. Dog Example – Understanding Variance

Four dog breeds → Great Dane, Greyhound, Golden Retriever, Jack Russell.

• Each breed ≈ similar within itself, different between breeds.

• Model: Breed explains some variance in weight.

• Error: Within-breed differences (e.g., age, diet).

4. Why Not Just Use Many t-Tests?

• Each t-test has 5 % chance of Type I error (α = 0.05).

• Multiple pairwise comparisons → compound error (“family-wise error rate”).

• With 6 possible pairings → chance of no error drops to ≈ 74 %.

  • Error probability ≈ 1 in 4 → too high.

• ANOVA keeps overall error rate at 5 % by testing all groups simultaneously.

5. General Linear Model (GLM) Concept

All parametric tests derive from GLM:

Outcome=Model Effect+Error\text{Outcome} = \text{Model Effect} + \text{Error}Outcome=Model Effect+Error

Example: Dog Weight = Breed Effect + Residual Error

6. Assumptions of One-Way ANOVA

7. The F Statistic

Definition

Ratio of variance explained by the model to unexplained variance:

F=Between-Group VarianceWithin-Group VarianceF = \frac{\text{Between-Group Variance}}{\text{Within-Group Variance}}F=Within-Group VarianceBetween-Group Variance​

• Large F → model explains much of the variance (significant).

• F ≈ 1 → model explains little (nonsignificant).

Interpretation

8. Understanding the ANOVA Table

Typical output:

Key:

• SS (Sum of Squares) = total variance measure.

• MS (Mean Square) = SS ÷ DF.

• F = MS_between ÷ MS_within.

• Two degrees of freedom reported: (df₁, df₂).

9. Hand-Calculation Demonstration (Excel Example)

Scenario

Clinic wants to know usage frequency of 3 speech tests over 10 weeks.
Each week = data point → 30 total observations.

Step Summary

1. Compute grand mean (≈ 8).

2. Find Total SS = 410.

3. Compute Within-Group SS = 316.4.

4. Compute Between-Group SS = 93.6.
    → 410 ≈ 93.6 + 316.4 ✔

5. Degrees of freedom: df_between = k – 1 = 2; df_within = N – k = 27.

6. MS_between = 93.6 / 2 = 46.8; MS_within = 316.4 / 27 = 11.72.

7. F = 46.8 / 11.72 = 3.99.

8. Critical F(2,27, α = 0.05) = 3.35 → 3.99 > 3.35 ✅ significant.

Interpretation:
Not all speech tests were used equally often (p < .05).

10. Running ANOVA in Minitab

Steps

1. Stat → ANOVA → One Way.

2. Choose “Response data in a separate column for each factor level.”

3. Select test columns and run.

Output (Example)

• F(2, 27) = 3.99, p = .03.

• R² = 23 % → model explains 23 % of variance (weak-moderate).

Interpretation

✅ p < .05 → Significant.
❌ Cannot yet tell which tests differ → need post-hoc (Tukey).

11. Reporting Conventions (APA Style)

12. Post-Hoc Analysis – Tukey HSD Test

Purpose

• Determines where significant differences lie after significant ANOVA.

• Controls family-wise error rate.

Minitab Procedure

Stat → ANOVA → One Way → Comparisons → Select “Tukey.”

Outputs

1. Interval Plot (Green) – visual means + CI; overview only.

2. 95 % Pairwise Comparison Plot (Orange) – CI for each pair:
    - If CI includes 0 → no significant difference.
    - If CI excludes 0 → significant difference.

3. Grouping Table (Letters Output) – easiest to interpret:
    - Groups sharing a letter = not significantly different.
    - Groups with no shared letters = significantly different.

Example

→ Test 3 and 2 do not share a letter → significantly different.

13. Applying to Dog Weights (4 Breeds)

• F(3, 36) = 477, p < .001 → highly significant.

• R² = 97.5 % → breed explains almost all weight variance.

• Tukey test shows:

  • Great Danes ≫ others (heaviest).

  • Jack Russells ≪ others (lightest).

  • Greyhounds ≈ Golden Retrievers (no difference).

APA Example

14. Practical Exercises

(a) Noise-Induced Hearing Loss by Occupation

• Groups: Dentists, Metalworkers, Salespeople, Truck Drivers.

• DV: 4 kHz Hearing Threshold.

• ANOVA significant → Tukey:

  • Metalworkers = poorest hearing.

  • Dentists < Salespeople.

  • Truck Drivers ≈ both Dentists and Salespeople.

• R² ≈ 79 % variance explained by occupation.

(b) Chocolate Weights

Two predictors tested: Brand and Type.

• “Chocolate type” explains more variance in weight than brand.

• Total SS same (425.6) but partitioned differently between model and error.

15. Concept Check / Quiz Summary

16. Key Takeaways

✅ Use ANOVA when comparing 3 or more group means.
✅ Assumptions: normal distribution + equal variances.
✅ Report: F(df₁, df₂) = …, p = …, include post-hoc if significant.
✅ Tukey test identifies which groups differ.
✅ R² = proportion of variance explained by model.
✅ Larger F and R² → stronger relationship between factor and outcome.