Post Hoc Comparisions in ANOVA: A Detailed Overview

Following Up in ANOVA

A significant F ratio in ANOVA only indicates that there are statistically meaningful differences somewhere among the groups if the independent variable has more than two levels.
It doesn't pinpoint which groups are significantly different from each other.
Therefore, follow-up comparisons are necessary to examine the differences between the means of specific groups of interest.

Family-wise Type I Error: The core issue when conducting multiple tests.
- The alpha rate (typically 0.05) represents the probability of making a Type I error (incorrectly rejecting the null hypothesis) for each test.
- Performing multiple tests on the same data increases the overall risk of making at least one Type I error.
- The more tests conducted, the greater the likelihood of finding a statistically significant difference simply by chance.
Example Scenario:
- A student analyzes a 20-item scale, performing individual t-tests on each item.
- Even if there's no real effect, one or more items might appear significant due to chance.
- If you conduct 20 significance tests, you would expect at least one of them to be statistically significant.
Diet Example:
- Comparing three diet groups (fruit, veggie, donut) requires three comparisons: fruit vs. veggie, fruit vs. donut, and veggie vs. donut.
- Running multiple t-tests without correction inflates the Type I error rate (e.g., to approximately 0.15), increasing the risk of falsely concluding there's a real effect.

Post Hoc Procedures (Discussed in this video):
- Used when there's no strong theoretical basis to expect specific differences between groups.
- The typical application of post hoc tests is when you expect that a diet affects happiness but you don't have specific pattern.
- It's acceptable to have a general hypothesis (e.g.,