week8.3_ANOVA

1. ANOVA Overview

  • Developed by L. H. Huhuhu, PhD, Assistant Professor of Business Administration at Washington and Lee University.

2. Example Scenario

  • An airline aims to determine which sound keeps callers on hold the longest:

    • (a) Advertisement about the airline

    • (b) Muzak

    • (c) Classical music (e.g., Vivaldi’s Four Seasons)

    • Rationale: Callers on hold lead to potential customer retention.

3. Conducting the Experiment

  • A randomized experiment played:

    • (a) Advertisement

    • (b) Muzak

    • (c) Classical music

  • Callers' hold times were recorded for analysis.

4. Issue with Multiple t-Tests

  • Inappropriate to use two-sample t-tests for:

    • Advertisement vs. Muzak

    • Advertisement vs. Classical

    • Classical vs. Muzak

    • Reason: Increases the risk of Type I error.

5. Confusion Matrix and Errors

  • Confusion Matrix Components:

    • True Positive: Correctly identifies a condition (e.g., pregnancy).

    • False Positive: Incorrectly identifies a condition.

    • Type I Error: Rejecting a true null hypothesis.

    • Type II Error: Failing to reject a false null hypothesis.

6. Type I and Type II Errors

  • When null hypothesis is true or false:

    • Type I Error (α): Rejecting null when true.

    • Type II Error (β): Failing to reject null when false.

    • Correct decision probabilities are equal to 1 - α and 1 - β, respectively.

7. Controlling for Confidence with Multiple Comparisons

  • Increased comparisons raise the error risk:

    • Example: If g = 10, there are 45 pairs for comparison.

    • Error probability per comparison leads to higher average incorrect intervals.

8. Analysis of Variance (ANOVA)

  • Hypothesis Testing:

    • Null Hypothesis (H0): Average hold times are equal (μ1 = μ2 = μ3).

    • Alternative Hypothesis (H1): At least two population means are different.

9. F-Test

  • F-statistic formula:

    • F = Between-group variability / Within-group variability

    • Larger variability between groups implies a larger F test statistic.

10. Degrees of Freedom

  • Calculating degrees of freedom:

    • Between-groups df1 = g - 1

    • Within-groups df2 = N - g (N = total observations; g = groups)

11. Example Calculation

  • For 3 groups and 15 total samples:

    • df1 = 3 - 1 = 2

    • df2 = 15 - 3 = 12

12. F-Value Calculation

  • F Calculation Steps:

    • Between Group Mean Square = Between sum of squares / (g-1)

    • Within Group Mean Square = Within sum of squares / (N-g)

13. ANOVA Table Structure

  • Standard metrics included in an ANOVA table:

    • Source | Sum of Squares (SS) | Degrees of Freedom (DF) | Mean Square (MS) | F-value | p-value | Effect Size (np2)

  • Values guide the conclusion on the null hypothesis.

14. Interpretation of Results

  • If p-value < 0.05, reject H0, implying the effects of the sounds are not equal.

15. Assumptions for ANOVA

  • Independence Assumption for groups and data.

  • Equal Variance Assumption: Works well if largest/ smallest standard deviation ratio <2.

  • Normality Assumption: Less of a concern with larger samples.

16. Post-Hoc Tests

  • Purpose: Multiple comparisons between groups (e.g., Advertisement vs. Muzak).

  • T-test and confidence intervals may be utilized.

17. Multiple Comparison Methods

  • Tailored to assess confidence levels for sets of comparisons, focusing on:

    • Bonferroni Method: Adjusts threshold p-value based on number of comparisons.

    • Tukey Method: Offers slightly narrower confidence intervals, potentially more complex.

18. Example Outputs from Tukey Method

  • Direct comparisons yield p-values indicating significance of group differences:

    • Comparisons include pairs such as C–A, M–A, M–C, etc.