Multiple groups design and ANOVA + Assumptions + Planned comparisons and post hoc tests

multiple groups design

IV has over 2 levels so over 2 groups to compare

can be between/within groups

ANOVA

determine if there’s a significant difference between the means of 3+ independent groups

compares variability between groups to variability within groups

tells us there’s a difference somewhere between means

F statistic

measures ratio of explained to unexplained variance

shape of F distribution changes depending on DF but always extreme positive skew

significant f depends on sample size and no. of conditions

always larger in RM design as smaller error term removes variability

Assumptions of ANOVA

DV is a continuous variable on a metric scale, independence of observations, normality of distributions, homogeneity of variance

independence assumption

not possible to predict one score from any other score. each participant’s score is independent and random assignment to groups with random selection of participants, each participant contributes only one score

normality assumption

samples are from normally distributed population, error is normally distributed between levels of IV

ANOVA mostly robust to breaches if kurtosis is similar, similar n and minimum 10-12 participants per condition

outliers

extreme score at one/both ends of distribution

causes violation of homogeneity of variance and increases Type I error

determine reason for being outlier then remove from data, use non parametric test, transform data

homogeneity of variance

variance of data within groups should be equal, and largest variance shouldn’t be more than 4x smallest variance

Levene’s test to check for breaches, then use robust tests like Welch and Brown-Forsythe, non parametric tests, lower alpha level

comparisons

tells us where the difference between means is

A priori planned comparisons

specific hypothesis for certain groups, done before data is collected

simple and complex

simple planned a priori comparisons

compares one group to another group

complex a priori comparisons

comparing set of groups to another set of groups

Assumptions of a priori planned comparisons

same as overall ANOVA especially homogeneity of variance

post hoc comparisons

compares all groups to each other to explore differences

find which specific groups differ significantly when overall test indicates significant difference

more exploratory

done after study

assumptions of post hoc comparisons

same as overall ANOVA + lack of planned comparisons

orthogonal contrasts

breaks down total variation to independent components, allowing to see individual effects of each condition

each contrast tests something different to other contrasts and accounts for all group differences

factors contributing to anova power

sample size, effect size (higher alpha = more power but also more risk of type 1 error), variability, research design

eta squared

tells us strength of treatment effect, ranges from 0-1

Type I error rates

use Bonferroni adjusted a level to account for many tests required and high type I error rates