Introduction to Independent Measures ANOVA and the Logic of Variance Analysis of Variance

Introduction to ANOVA

• Definition and Scope: ANOVA stands for Analysis of Variance. It is a statistical method used to test for significant differences among three or more sample means.

• Context in Course: This material is introduced in Chapter 12. It follows the progression from one-sample t-tests (Chapter 8 and 9) and two-sample t-tests (Chapter 10 and 11).

• Flexibility: Unlike t-tests, which are limited to two samples at a time, ANOVA is "extremely freeing" because it can handle any number of samples (e.g., comparing 11 different drug doses simultaneously).

• Instructional Note: The instructor emphasizes that this topic is typically broken into three parts. Watching all parts is mandatory for comprehension; missing the third segment (which usually contains the worked example) renders the first two segments useless for practical application.

Comparison Between ANOVA and T-Tests

• Reliance on Difference (T-tests): T-tests are built around the concept of subtraction. The formulas (e.g., $M - \nu$ or $M_1 - M_2$) focus on finding the literal difference between two values. Because subtraction is inherently a binary operation (comparing one thing to another), t-tests are limited to comparing only two groups at once.

• Reliance on Variance (ANOVA): ANOVA takes a completely different approach. It does not use subtraction; instead, it uses a ratio of variances.

• The Ratio Concept: A ratio is similar to a fraction where one number is divided by another. In ANOVA, the calculation involves dividing one variance by another variance.

• Data Shift: While t-tests focus on sample means (Chapter 3 concepts), ANOVA focuses on variability (Chapter 4 concepts).

The Limitations of Multiple T-Tests

One might ask why we cannot simply use multiple t-tests to compare three or more groups (e.g., comparing Group A to B, B to C, and A to C). There are two primary reasons why this is avoided:

• Practical Reason (Growth of Comparisons): The number of required t-tests increases rapidly as the number of sample means ($k$) increases:

  • 3 Sample Means: Requires 3 t-tests (A-B, B-C, A-C).

  • 4 Sample Means: Requires 6 t-tests.

  • 6 Sample Means: Requires 15 t-tests.

• Organizational Burden: Even using software like SPSS, keeping track of 15 different t-tests to analyze a single experiment is a "nightmare" and inefficient.

Type I Error Inflation

• The Statistical Conflict: Using multiple hypothesis tests to analyze a single experiment leads to Type I Error Inflation.

• Defining Type I Error: A Type I error is a false positive. It occurs when a researcher concludes that a treatment has an effect when it actually does not.

• Alpha ($\alpha$): The probability of making a Type I error on any single hypothesis test is equal to the alpha level set by the researcher ($\text{Probability of Type I Error} = \text{Alpha level}$).

  • If $\alpha = 0.01$, there is a 1% chance of a false positive.

  • If $\alpha = 0.05$, there is a 5% chance of a false positive.

• The Accumulation Effect: Alpha is set per hypothesis test. If an experiment requires multiple tests (like the 3 tests needed for 3 samples), the 5% risk of error exists for each test independently. This means the overall probability of making at least one false positive error in the entire experiment rises significantly above 5%.

• Mathematical Probability: While not a simple addition (5% + 5% + 5% = 15%), the cumulative error for three tests at $\alpha = 0.05$ is approximately $14.7\%$. This level of risk is considered unacceptable in scientific research.

Terminology in ANOVA

ANOVA uses specific terminology for components researchers have previously studied under other names:

• Factor: The independent variable (IV) or quasi-independent variable (e.g., "Dose of Drug").

• Levels: The specific groups or conditions that make up the factor.

  • Example: If a study tests 0mg, 50mg, 100mg, and 200mg of a drug, the Factor is "Dose" and there are four levels.

• Dependent Variable (DV): The actual measurement taken from participants (e.g., memory scores, depression scores, or pain levels).

Experimental Designs in ANOVA

Just like t-tests, ANOVA can be applied to different types of experimental designs:

• Independent Measures ANOVA (Chapter 12): Every sample of scores comes from a completely different group of people. If there are four levels with 25 people in each, the study requires 100 different participants.

• Repeated Measures ANOVA (Chapter 13): The same group of individuals is used in every level of the factor.

  • Example: Measuring the same 25 people at three time points (Before, During, After treatment). This study only requires 25 participants total.

The Logic of ANOVA: Sources of Variability

ANOVA works by analyzing where the variability in scores comes from. There are two primary sources:

• Within-Group Variability: This refers to the differences between scores within the same level/group.

  • Example: Two students at the same school (School 1) have different test scores.

  • Cause: Since they are at the same school, the school cannot be the reason for their difference. The variability is caused by individual differences or random error.

• Between-Group Variability: This refers to the differences between the means of the different groups/levels.

  • Example: Comparing the average score of School 1, School 2, and School 3.

  • Cause: If the means are different, part of that variability may be caused by the "Factor" (the school they attend).

Analyzing the Source of the Effect

• Treatment Effect: If a factor (like a drug dose or a specific school) has an actual effect, that effect will manifest as between-group variability. It makes the group means more different from one another.

• Realistic Expectations: In any real-world study, you will have both within-group and between-group variability simultaneously.

• Hypothetical Scenarios for Visualization:

  • Variability Between but Not Within: Every student within a school gets the exact same score, but different schools have different averages (e.g., all 3s at School 1, all 4s at School 2, all 5s at School 3). This is unrealistic but shows purely between-group variance.

  • Variability Within but Not Between: Students within a school have different scores, but every school has the exact same average (e.g., Mean = 4 for all schools). Here, there is no "Factor effect" or treatment effect; the differences are purely within-group.

• Conclusion: To determine if a factor (the school, the drug) works, ANOVA looks at whether the differences between the groups are significantly larger than the differences within the groups.