PSY 2030 One-Way Between-Groups ANOVA: Formulas, F Tests & Post Hoc Comparisons

ANOVA

Analysis of Variance; a statistical test that compares the variance between groups to the variance within groups.

ONE-WAY ANOVA

An ANOVA with ONE independent variable.

BETWEEN-GROUPS ANOVA

An ANOVA comparing separate groups of people.

WHEN DO YOU USE A ONE-WAY BETWEEN-GROUPS ANOVA?

When you have one independent variable with multiple levels/groups and want to determine whether the population means differ.

ANOVA VS. t TEST

A t test can compare two means, while ANOVA is used to compare multiple groups/levels of one independent variable.

WHY NOT JUST RUN MULTIPLE t TESTS?

Because conducting multiple tests increases the overall probability of making at least one Type I error.

EXAMPLE — 3 GROUPS

A researcher wants to compare three different therapies. Instead of conducting three separate t tests, use a one-way ANOVA.

EXAMPLE — 4 GROUPS

A researcher wants to compare four different therapies. A one-way ANOVA can test all four groups together.

ONE-WAY MEANING

"One-way" means there is one independent variable.

BETWEEN-GROUPS MEANING

"Between-groups" means the analysis compares separate groups of people.

TYPE I ERROR

Rejecting the null hypothesis when the null hypothesis is actually true.

ALPHA (α)

The probability of committing a Type I error that the researcher is willing to accept.

NOMINAL ALPHA

The alpha level that we specify for a test, such as α = .05.

α = .05

An accepted nominal Type I error rate of 5% for the specified comparison/family of hypotheses.

TRUE ALPHA

The actual Type I error rate, which may differ from the nominal alpha.

MULTIPLE COMPARISONS

Performing multiple statistical comparisons involving the same family of hypotheses/data.

WHY MULTIPLE COMPARISONS ARE A PROBLEM

Each additional comparison creates another opportunity to make a Type I error, increasing the overall familywise error rate.

3 GROUPS — NUMBER OF PAIRWISE COMPARISONS

If there are 3 groups, there are 3 pairwise comparisons.

3 GROUPS — COMPARISONS

1 vs. 2, 1 vs. 3, and 2 vs. 3.

4 GROUPS — NUMBER OF PAIRWISE COMPARISONS

If there are 4 groups, there are 6 pairwise comparisons.

4 GROUPS — COMPARISONS

1 vs. 2, 1 vs. 3, 1 vs. 4, 2 vs. 3, 2 vs. 4, and 3 vs. 4.

WHY DOES α INCREASE WITH MULTIPLE TESTS?

Each test provides another opportunity to make a Type I error, so the probability of making at least one Type I error across the family increases.

PROBABILITY OF NOT MAKING A TYPE I ERROR

1 − α.

EXAMPLE — TWO COMPARISONS WITH α = .05

Probability of no Type I error for each comparison = .95. Probability of no Type I error across both = .95 × .95 = .9025.

TRUE ALPHA FOR TWO COMPARISONS

1 − .9025 = .0975, or 9.75%.

IMPORTANT MULTIPLE-COMPARISON IDEA

Using α = .05 for each of several tests does NOT mean the overall probability of making at least one Type I error remains exactly 5%.

BONFERRONI CORRECTION

A method for adjusting alpha when conducting multiple comparisons.

BONFERRONI FORMULA

αadjusted = α / n.

IN THE BONFERRONI FORMULA, WHAT DOES α REPRESENT?

The original/nominal alpha level.

IN THE BONFERRONI FORMULA, WHAT DOES n REPRESENT?

The number of comparisons being conducted.

BONFERRONI EXAMPLE — 2 COMPARISONS

If α = .05 and there are 2 comparisons: .05 / 2 = .025.

BONFERRONI EXAMPLE — 3 COMPARISONS

If α = .05 and there are 3 comparisons: .05 / 3 ≈ .0167.

BONFERRONI EXAMPLE — 6 COMPARISONS

If α = .05 and there are 6 comparisons: .05 / 6 ≈ .00833.

WHY USE BONFERRONI?

To make the individual tests more conservative so the overall true/familywise alpha stays at or below the desired level.

BONFERRONI TRADEOFF

The correction reduces Type I error but can make it harder to detect real effects, reducing statistical power.

FAMILYWISE ERROR RATE

The probability of making at least one Type I error across a family of statistical tests.

FAMILY OF TESTS

A set of related statistical tests/comparisons addressing the same general set of hypotheses.

WHY CORRECT WITHIN A FAMILY?

Because the tests within a family use the same data and typically concern related hypotheses.

F DISTRIBUTION

The sampling distribution of the ratio of two variances.

WHAT DOES AN F STATISTIC REPRESENT?

A ratio comparing one variance to another variance.

F DISTRIBUTION DEGREES OF FREEDOM

The F distribution has TWO degrees of freedom: one for the numerator and one for the denominator.

df₁

Numerator degrees of freedom.

df₂

Denominator degrees of freedom.

F(df₁, df₂)

The F distribution specified by numerator degrees of freedom and denominator degrees of freedom.

WHY DOES F HAVE TWO dfs?

Because an F statistic is a ratio of two variance estimates, each with its own degrees of freedom.

F VALUES

F values are positive.

CRITICAL F VALUE

An F value used as the cutoff for determining whether the observed F is statistically significant.

BETWEEN-GROUPS VARIANCE

Variance associated with differences among the group means.

WITHIN-GROUPS VARIANCE

Variance associated with differences among individual scores within each group.

ANOVA'S BASIC IDEA

ANOVA compares between-groups variance with within-groups variance.

ANOVA F RATIO

F = Between-Groups Variance / Within-Groups Variance.

WHAT DOES A LARGE F SUGGEST?

A large F means the between-groups variance is large relative to the within-groups variance, providing evidence that the group means may differ.

WHAT DOES F ≈ 1 MEAN?

If between-groups variance is about the same as within-groups variance, F is approximately 1 and there is little evidence of a group difference.

F = 1

Occurs when between-groups variance and within-groups variance are equal.

WHY DOES ANOVA USE VARIANCE?

ANOVA determines whether differences among group means are large relative to the variability of individual scores within groups.

GRAND MEAN

The mean of ALL scores in the total sample.

GRAND MEAN FORMULA

GM = ΣX / N.

WHAT DOES GM REPRESENT?

The overall mean when every score from every group is combined.

WHAT DOES N REPRESENT IN GM = ΣX/N?

N = total number of scores across all groups.

GROUP MEAN

The mean of the scores within one particular group.

GROUP MEAN FORMULA

M = ΣX / n.

WHAT DOES M REPRESENT?

The mean of a particular group.

WHAT DOES n REPRESENT IN THE GROUP MEAN FORMULA?

n = number of scores in that particular group.

DIFFERENCE BETWEEN n AND N

Lowercase n refers to the number of scores in a particular group; uppercase N refers to the total number of scores across all groups.

Degrees of freedom between-groups

dfbetween = k − 1.

Degrees of freedom within-groups

dfwithin = N − k.

Total degrees of freedom

dftotal = N − 1.

What does k represent?

The number of levels/groups of the independent variable.

What does N represent in dfwithin = N − k?

The total number of observations/scores across all groups.

What does N represent in dftotal = N − 1?

The total number of observations/scores across all groups.

What does k represent in dfbetween = k − 1?

The total number of groups/levels of the independent variable.

DF Check

dfbetween + dfwithin = dftotal.

DF Check Example

If dfbetween = 2 and dfwithin = 12, then dftotal should equal 14.

Sum of squares — between groups

SSbetween represents variation among the group means around the grand mean.

SSbetween formula

SSbetween = Σn(M − GM)².

What does n represent in SSbetween?

The number of scores in that particular group.

What does M represent in SSbetween?

The mean of the particular group.

What does GM represent in SSbetween?

The grand mean of all scores.

What does (M − GM) represent?

The difference between a group's mean and the overall grand mean.

What does (M − GM)² represent?

The squared difference between a group mean and the grand mean.

Why multiply by n?

Each group's deviation from the grand mean is weighted by the number of scores in that group.

Mean square — between groups

MSbetween is the between-groups sum of squares divided by the between-groups degrees of freedom.

MSbetween formula

MSbetween = SSbetween / dfbetween.

What does MS mean?

Mean square.

What does MSbetween represent?

The estimated between-groups variance.

Sum of squares — within groups

SSn is the sum of squared deviations of scores from their own group's mean for a particular group.

SSn formula

SSn = Σ(Xn − Mn)².

What does Xn represent?

An individual score from a particular group.

What does Mn represent?

The mean of that particular group.

SSwithin

The total within-groups sum of squares.

SSwithin formula

SSwithin = SS1 + SS2 + ... + SSn.

How to calculate SSwithin

Calculate SS separately for every group, then add all of the group SS values together.

Mean square — within groups

MSwithin is the within-groups sum of squares divided by the within-groups degrees of freedom.

MSwithin formula

MSwithin = SSwithin / dfwithin.

What does MSwithin represent?

The estimated within-groups variance.

F statistic

The ratio of the between-groups mean square to the within-groups mean square.

F formula

F = MSbetween / MSwithin.

What does the numerator of F represent?

Between-groups variance, represented by MSbetween.

What does the denominator of F represent?

Within-groups variance, represented by MSwithin.

F formula using variance

F = Between-Groups Variance / Within-Groups Variance.