Module Three: Multiple Comparisons

ANOVA is an Omnibus test, which means that it tells us that there is a difference but not where the difference is.

Further analysis is needed to determine where the difference is

Family-wise error rate is the probability of making one or more false discoveries, or type I errors when performing multiple hypotheses tests.

Planned Comparisons are A Priori. they test specific hypothesises, which are proposed prior to data collection

Types of Planned Comparisons

Orthogonal Contrasts compare unique “chunks” of variance. They use Helmert and Difference comparisons.

Helmert - Compare each category to the mean of subsequent categories

Difference - Compare each category to the mean of previous categories

Non-orthogonal Contrasts overlap or use the same “chunks” of variance in multiple comparisons. They require careful interpretation and lead to increased type 1 error rate

•       Non-Orthogonal: Deviation, Simple, Repeated

Polynomial Contrasts

•       Linear, Quadratic, Cubic and Quartic trends

polynomial contrasts are only used when IV is ordinal

Post Hoc compare all groups with a stricter alpha value, and hypothesis formed after data collection

The simplest Post Hoc is the Bonferroni test

•       Tukey’s HSD

•       Called Tukey’s HSD (Honestly Significant Difference)

•       The cumulative probability of a type 1 error never exceeds the specified level of significance (p < .05)

•       Supplies a single critical value (HSD) for evaluating the ‘significance’ of each pair of means

•       The critical value (HSD) increases with  (i.e., each additional group mean)

•       It becomes more difficult to reject the null hypothesis as a greater number of group means are compared

•       If the absolute (i.e., obtained) difference between two means exceeds the critical value for HSD, the null hypothesis for that pair of means can be rejected