Module Three: Multiple Comparisons
ANOVA is an Omnibus test, which means that it tells us that there is a difference but not where the difference is.
Further analysis is needed to determine where the difference is
Family-wise error rate is the probability of making one or more false discoveries, or type I errors when performing multiple hypotheses tests.
Planned Comparisons are A Priori. they test specific hypothesises, which are proposed prior to data collection
Types of Planned Comparisons
Orthogonal Contrasts compare unique “chunks” of variance. They use Helmert and Difference comparisons.
Helmert - Compare each category to the mean of subsequent categories
Difference - Compare each category to the mean of previous categories
Non-orthogonal Contrasts overlap or use the same “chunks” of variance in multiple comparisons. They require careful interpretation and lead to increased type 1 error rate
• Non-Orthogonal: Deviation, Simple, Repeated
Polynomial Contrasts
• Linear, Quadratic, Cubic and Quartic trends
polynomial contrasts are only used when IV is ordinal
Post Hoc compare all groups with a stricter alpha value, and hypothesis formed after data collection
The simplest Post Hoc is the Bonferroni test
• Tukey’s HSD
• Called Tukey’s HSD (Honestly Significant Difference)
• The cumulative probability of a type 1 error never exceeds the specified level of significance (p < .05)
• Supplies a single critical value (HSD) for evaluating the ‘significance’ of each pair of means
• The critical value (HSD) increases with (i.e., each additional group mean)
• It becomes more difficult to reject the null hypothesis as a greater number of group means are compared
• If the absolute (i.e., obtained) difference between two means exceeds the critical value for HSD, the null hypothesis for that pair of means can be rejected
ANOVA is an Omnibus test, which means that it tells us that there is a difference but not where the difference is.
Further analysis is needed to determine where the difference is
Family-wise error rate is the probability of making one or more false discoveries, or type I errors when performing multiple hypotheses tests.
Planned Comparisons are A Priori. they test specific hypothesises, which are proposed prior to data collection
Types of Planned Comparisons
Orthogonal Contrasts compare unique “chunks” of variance. They use Helmert and Difference comparisons.
Helmert - Compare each category to the mean of subsequent categories
Difference - Compare each category to the mean of previous categories
Non-orthogonal Contrasts overlap or use the same “chunks” of variance in multiple comparisons. They require careful interpretation and lead to increased type 1 error rate
• Non-Orthogonal: Deviation, Simple, Repeated
Polynomial Contrasts
• Linear, Quadratic, Cubic and Quartic trends
polynomial contrasts are only used when IV is ordinal
Post Hoc compare all groups with a stricter alpha value, and hypothesis formed after data collection
The simplest Post Hoc is the Bonferroni test
• Tukey’s HSD
• Called Tukey’s HSD (Honestly Significant Difference)
• The cumulative probability of a type 1 error never exceeds the specified level of significance (p < .05)
• Supplies a single critical value (HSD) for evaluating the ‘significance’ of each pair of means
• The critical value (HSD) increases with (i.e., each additional group mean)
• It becomes more difficult to reject the null hypothesis as a greater number of group means are compared
• If the absolute (i.e., obtained) difference between two means exceeds the critical value for HSD, the null hypothesis for that pair of means can be rejected