STATS Introduction to ANOVA

Problem with multiple independent (t-)tests

• Capitalising on chance = cumulating probability/chance of an error

  • If the .05 level of significance is adopted, the 5% chance of an error accumulates across the multiple tests (multiple tests being the DV across the different levels of the IV) → leading to a higher Type I error rate

• Type I error (false positive)

• Type II error (false negative)

Cumulative probability equation (Capitalising on chance)

Example: there are 6 separate independent t-tests (Each level of the IV: vocabulary learning method)

• L1 direct translation vs L2 definitions 

• L1 direct translation vs Loci method 

• L1 direct translation vs Reminiscence 

• L2 definitions vs Loci method 

• L2 definitions vs Reminiscence 

• Loci method vs Reminiscence 

= 1 - (1 - a)ⁿ Where n is the number of tests and a is the level of significance

= 1 - (1 - .05)⁶

= 0.26

Therefore, by doing 6 different t-tests you now have 26% of risk that you will make Type I error instead of the original 5% from .05

One-way between-subjects ANOVA

Only one IV (vocabulary learning method)

Each participant appears under only one level/condition

Difference between between-subjects ANOVA and T-tests

ANOVA (Analysis of Variance)

• Compares means between 3 or more groups

• Uses Variance to measure the differences (More variation = more difference between groups, no variation = no difference)

  • Between-groups variation vs within-group variation

T-tests

• Compares means between 2 groups

• Leads to: student’s t-test (assumes equal variance) or Welch’s t-test (does not assume equal variance)

• Non-parametric alternative: Mann-Whitney U-test

• t = obtained difference between two sample means / standard error

Between-group variation

Mean of each level of IV is calculated

Variation BETWEEN the means of groups is looked at (how much the group averages differ from one another)

Measures effect of error and treatment

Factors leading to between-group variation

Treatment effects

• Each group uses a different vocabulary learning method; if some learn/remember better, the average vocabulary scores of the group will be different

Error

• Variation could be due to chance rather than IV effect

• Error in ANOVA = individual differences + random factors

  • Some may be stronger learners, tired, distracted

  • Small differences in testing conditions

Within-group variation

Comparing how individual participant scores differ within each level of IV rather than the mean of each level

Measures effect of error

Causes of within-group variation

Error in ANOVA = individual differences + random factors

• Some learners differ in ability despite the same technique, due to different memory ability, prior English knowledge and motivation

• Random factors e.g. being tired, misunderstanding the question or the testing environment is noisy

Total variance

Between-group variance + within-groups variance = total variance (overall variation observed in the data)

ANOVA looks at the partition of total variation

Partition BREAKS DOWN the total variation observed the data into components

How to calculate Total variance

1. Calculate the mean for each group

2. Calculate the grand mean (sum of all individual scores from all groups, divide by number of observations) e.g. 4400/60 = 73.33

3. Calculate total variance of scores around sample means (within-groups variance) - how much do individuals differ from their own group mean?

4. Calculate variance of sample means around grand means (between-groups variance) - how much do the group means differ from the grand means?

F-ratio: the test statistic for ANOVA

F = Between-groups variance / Within-groups variance = treatment effect + error / error

Can vary from 0 to infinity

If the between-group variance > within-group variance, it means:

• The F-value is large

• The observed differences among groups means are unlikely to be due to chance alone