ANOVA Notes

ANOVA: A Conceptual Approach

Introduction to ANOVA

• ANOVA (Analysis of Variance) is used when an experimenter randomly assigns people to more than two groups to study the effect of a manipulation on a continuous outcome.
• It's a significance test used in situations with more than two groups.

ANOVA Example: Diet and Happiness

• Research Question: Does people's diet affect their happiness?
• Study Design:
  • A sample of 60 people is taken and randomly assigned to one of three groups (20 people per group):
    • Fruit group: Eat at least three fruits a day.
    • Veggie group: Eat at least three vegetables a day.
    • Donut group: Eat at least three donuts a day.
  • Happiness is measured on a continuous scale.
• Independent Variable: Categorical (diet groups: fruit, veggie, donut).
• Dependent Variable: Continuous (happiness score).

Possible Outcomes and Hypotheses

• Sample Means (Hypothetical):
  • Fruit group mean: $\bar{x}<em>1 = 3$ (estimated population variance $\hat{\sigma}^2</em>1 = 1$)
  • Veggie group mean: $\bar{x}<em>2 = 3.2$ (estimated population variance $\hat{\sigma}^2</em>2 = .9$)
  • Donut group mean: $\bar{x}<em>3 = 4.0$ (estimated population variance $\hat{\sigma}^2</em>3 = 1.2$)
• Two Possibilities:
  • Null Hypothesis: The differences between groups are due to sampling error, meaning the three samples are drawn from the same population (no real effect of diet).
  • Alternative Hypothesis: The differences between groups are due to the effect of diet, meaning the samples are drawn from populations with different means.

Evaluating the Null Hypothesis

• To evaluate the null hypothesis, we need to know how likely it is to observe the differences we observed if the null hypothesis is true.

The F-Ratio

• In ANOVA, the test statistic is the F-ratio.
• The F-ratio is the ratio of the population variance as estimated between groups versus the population variance as estimated within groups.

Variation in Sample Means from the Same Population

• Anytime we take random samples of the same size from a population, we observe variation in the sample means, even if the samples come from the same population.
• This is due to sampling error: The sample is only a subset of the population and, hence, only represents a portion of the scores comprising the population.

Implications of Sampling Error

• We will observe variability in the sample means for the fruit, veggie, and donut groups, even if the null hypothesis is true.
• How much variability we observe depends on:
  • Sample size (N): As N increases, sampling error decreases (we'll ignore this factor for now).
  • Variance of the scores in the population: When there is a lot of variation in happiness in the population, we'll be more likely to observe variation among our sample means.

Estimating Variance in the Population

• The first step in ANOVA is to estimate the variance of the dependent variable (happiness) in the population.
• We'll use this information to determine whether the variation in sample means that we observe is greater than what we would expect if the samples all came from the same population.
• We'll estimate this variance in two ways: within groups and between groups.

Method #1: Within Groups

• If the samples come from the same population, each of the three sample variances is an estimate of the same quantity (i.e., the population variance).
• Thus, we can average the three estimates of the population variance to get a single, pooled estimate of the population variance.

MSWithin

• In our example, we pool (i.e., average) 1, .9, and 1.2.
• (1 + .9 + 1.2)/3 = 1.03
• Our pooled estimate of the population variance is 1.03.
• In ANOVA-talk, this pooled estimate of the population variance is called Mean Squares Within or MSWithin.
• We use the term “within” because we are estimating the population variance separately within each sample or condition.
• Formula: $MS<em>{Within} = \frac{\sum \hat{\sigma}^2</em>{\text{groups}}}{N}$

Method #2: Between Groups

• There is another way to estimate the variance in the population, based on studying the variation in sample means across or between conditions.
• We can view our three sample (i.e., condition) means as constituting a “sampling distribution” of sorts based on three samples instead of an infinite number of samples (as is the case in a theoretical sampling distribution).
• Hence, if we calculate the variance among these three sample means, we're essentially estimating the variance of the sampling distribution of means.
• Using this information, we can get our second estimate of the population mean.

Sampling Distribution of Means

• Sampling distribution of means: A theoretical distribution of sample means based on an infinite number of samples drawn from a population of scores.
• Basic facts:
  • Mean of a sampling distribution equals the population mean (not relevant to this discussion).
  • Variance equals: $\frac{\sigma^2}{N}$ where $\sigma^2$ is the variance of the scores in the population.

Estimating Population Variance from Variance of Sample Means

• The variance of the sampling distribution of means provides an estimate of $\frac{\sigma^2}{N}$.
• If we can estimate $\frac{\sigma^2}{N}$, then we have an estimate of $\sigma^2$ (the population variance)!!!

Calculating Variance Among Sample Means

• We have three sample means.
• To find the variance among them, we find the average squared difference among them.
• Grand Mean (GM): The mean of the three means.
• GM = (3 + 3.2 + 4)/3 = 3.4

Estimating Variance of the Sampling Distribution

• We estimate the variance of the sampling distribution of means by studying the average squared deviation of these three means from the Grand Mean.
• We use 1 less than the number of groups in the denominator because because we are estimating a variance, albeit, the variance of a sampling distribution of means instead of a population variance.
• Any variance that is an estimate of a population variance will be a bit too small.
• Formula: $\hat{\sigma}^2<em>M = \frac{\sum(M - GM)^2}{N</em>{\text{groups}} - 1}$

Calculation Example

• $N<em>{\text{groups}} = 3$, so $N</em>{\text{groups}} – 1 = 2$
• $\sum(M – GM)^2 / 2 = (.16 +.04+.36)/2 = .28$

MSBetween

• We have an estimate of the variance of the sampling distribution: 0.28
• N (the sample size within a condition) is 20, so our estimate of the population variance is 5.6
• We estimate the population variance by multiplying the variance among means by the sample size.
• This quantity is called Mean Squares Between or MSBetween.
• $MS<em>{Between} = N \cdot \hat{\sigma}^2</em>M$
• Where N is the number of people in each group.
• $\hat{\sigma}^2 = N \cdot \hat{\sigma}^2_M = 20 \cdot 0.28 = 5.6$

Big Picture: Two Ways to Estimate Population Variance

• There are two ways to estimate the population variance from our three sample/condition means:
  • MSWithin: We can pool the variance estimates that are calculated within each condition.
  • MSBetween: We can treat the three sample means as an approximate sampling distribution of means. By estimating the variance of this sampling distribution, we can also estimate the variance of the population.

F Ratio

• If the null hypothesis is true, then these two mathematically distinct estimates of the population variance should be identical.
• If we express them as a ratio, the ratio should be close to 1.00.
• $F = \frac{MS<em>{Between}}{MS</em>{Within}}$

Interpreting the F Ratio

• If the null hypothesis is false, and we are sampling from populations with different means, the variance between means will be greater than what we would observe if the null hypothesis was true.
• The size of MSWithin, however, will be the same.
• Thus, the F ratio will become increasingly larger than 1.00 as the difference between population means increases.

F Ratio and Sampling Error

• Even if the null hypothesis is true, the F ratio will depart from 1 because of sampling error.
• The degree to which it departs from 1, under various sample sizes, can be quantified probabilistically (like all other test statistics we have discussed thus far).
• When the p-value associated with the F-ratio for a specific sample size is < .05, we reject the null hypothesis. When it is larger than .05, we accept the null hypothesis.

Degrees of Freedom

• To calculate the degrees of freedom for our test:
  • $df<em>{between} = N</em>{\text{groups}} – 1 = 3-1 = 2$ (Numerator df)
  • $df<em>{within} = df</em>1 + df<em>2 + … + df</em>{\text{last}} = (20-1) + (20-1) + (20 - 1) = 57$ (Denominator df)
• We could use these values to determine a “critical value” of F if we compute this by hand.
• $F_{critical} = \sim 3.17$

Example F Ratio Calculation and Interpretation

• In our example, MSWithin was 1.03. MSBetween was 5.6. Thus,
• $F = \frac{MS<em>{Between}}{MS</em>{Within}} = \frac{5.6}{1.03} = 5.4$
• Recall that $F_{critical} = 3.17$. Our calculated F exceeds this critical value; the p-value associated with this particular F is less than .05. Thus, we would reject the null hypothesis and conclude that diet has an effect on happiness.