Statistics Unit 4: One-Way ANOVA, Repeated Measures ANOVA, FACTORIAL ANOVA

What is ANOVA?

an analysis of variance between or within groups using sample data to make inferences about the population.
- an inferential statistics technique for comparing means, comparing variances, and assessing interactions

What does ANOVA help determine?

whether or not differences in sample means reflect sampling error or an effect caused by the manipulation of an IV

What is a One-Way ANOVA?

a test used to determine whether there are any statistically significant differences between the population means of two or more independent groups

When comparing more than two groups to one another, why wouldn't you do multiple t-tests over ANOVA?

1. They can be tedious. [ n(n-1)/2 ] the larger n, the more comparisons, and the more tests you'd have to run.
2. They increase our risk of a Type I error!

When is a one-way ANOVA appropriate?

- it is an independent-samples design
- there is only ONE (1) manipulated variable (IV)

What does it mean to say that we are trying to determine whether differences between groups are due to "noise" or a likely "real" effect?

ANOVA will help us determine whether or not differences in sample means reflect sampling error or an effect caused by the manipulation of an IV

What does it mean to say that the question we are trying to answer with ANOVA is about the ratio of between to within-group variation?

"Does the variation BETWEEN the groups exceed the variation WITHIN the groups?"

What are the two general categories of variability?

Between-Group and Within-Group

How can we describe Between-Group variability?

What difference is there between the independent groups in the analysis

How can we describe Within-Group variability?

What difference is there within each independent group in the analysis

What are the differences between Between-Group and Within-Group variability?

Between-Group variability may contain effects of the IV ("treatment effect")???

How does the F distribution differ from the t distribution?

F-dist. is a function of the df between groups (numerator) and the df within groups (denominator) while T dist is..?

What can we expect our test F value to look like when there is no effect of the IV (H0 is true)?

the overall value of the F ratio should be 1 or close to 1.

What can we expect our test F value to look like when the effect of the IV is large (H0 is false)?

the overall value of the F ratio will be larger than 1

What are the assumptions of ANOVA?

Normality - the DV is assumed to be normally distributed in the population from which the samples are drawn
Homogeneity of variance - the variance in DV scores are assumed to be equal for each population
Random assignment - the extraneous variables that could affect the study are equally distributed in each level of the IV

How do we conduct a NHST for a one-way ANOVA?

Step 1: Specify the test statistic and explain why it's the appropriate test
Step 2: State the null and alternative hypotheses
Step 3: Specify the sample size, # of IV levels, df, and locate CV
Step 4: Calculate your test statistics
Step 5: Make a decision about H0

What are the differences in a NHST for a one-way ANOVA versus NHST for other designs that we've learned?

- more than two samples
- F distribution
- Default nondirectional

What is Tukey's HSD?

a post hoc significance test that compares all possible pairs of treatments.

What does Tukey's HSD test tell us?

After a significant F statistic occurs, it tells us where the significant differences actually lie (which group means are significantly different).
- If HSD test value > HSD CV (or p < .05), then we have a significant difference.
- If HSD test value < HSD CV (or p > .05), then we do not have a significant difference.

How do you use Tukey's HSD to determine whether or not there's a significant difference?

Locate the number of levels of the IV (K) and the dferror.
Then, use Table G to find the critical value for the HSD tests

What does Tukey's HSD tell us?

whether or not one sample mean is significantly larger than the other.

What does it mean to call the one way ANOVA test a post hoc test?

It means it is a multiple comparison test done after examination

What does an effect size using Cohen's d tell us?

Effect size between groups

What does an effect size using n squared tell us?

Effect size for the overall ANOVA

How do you interpret an effect size using n squared?

Small ≈ .01
Medium ≈ .06
Large ≈ .14

What does a strong conclusion look like for a one-way ANOVA?

"A ___ ANOVA revealed the presence or absence of others does significantly affect the ability to solve a complex problem, F(2, 15) = 9.71, p = .002, Ƞ2 = .56. The magnitude of this effect was very large. !!Specifically, when three people were present, it took participants significantly longer to solve problems (M = 3.27) than when they were alone (M - 0.62), p = .002, d = -2.46, or when just one other person was present (M = 1.37), p = .02, d = -1.77. The magnitude of both of these effects was quite large. However, there was no significant difference in time taken to solve problems when one person was present versus being alone, p = .47, though the magnitude of this effect was moderately large, d = -.69!!"

What does a thorough conclusion of a one-way ANOVA have?

- A statement of the test used •
- The context of the research question •
- The results of the F test •
- The results of Tukey (HSD) tests •
- Statements of effect size

Conceptually, how does a one-way ANOVA differ from a repeated measures ANOVA?

- RM ANOVA has Within participant (subject) or within pair variability

In terms of calculations, how does a one-way ANOVA differ from a repeated measures ANOVA?

- RM ANOVA includes calculations of SSsubjects to partial out variability due to individual differences

What is within participant(subject) variability?

Variability within the scores of a single participant (subject). By calculating, we can more effectively factor out the effects of individual differences.

How does within participant(subject) affect error variability?

- Increases power! (Probability of Type II Error)

What does a good, thorough conclusion look like for a one-way ANOVA or a repeated measures ANOVA?

"According to a repeated-measures ANOVA, the type of test-taking instructions provided significantly affected exam performance, F(2, 6) = 13.00, p = .007, n2 = .02. The magnitude of this effect was very large."

When is factorial ANOVA appropriate?

is appropriate when we need to compare samples with two or more present IVs

How do we denote factorial designs in shorthand?

- # of Rows by # of Columns
- 2x2 is the most commo

What are the sources of variability in factorial ANOVA?

- Main effect of Factor A (Treatment)
- Main effect of Factor B (Treatment)
- Interaction of A x B (Treatment)
- Error

What is a main effect?

a test for significant differences between the main levels of one IV and the grand mean, ignoring the other IVs

How do we interpret a main effect?

You test for a main effect of each individual IV

What is an interaction?

a test to see if the effect of one IV on the DV depends on which level of the other IV is being administered

How do we interpret an interaction?

You test for interactions between two or more variables

What three statistical tests do we compute for a two-way factorial ANOVA?

- Test for main effect of Factor A
- Test for a main effect of Factor B
- Test for an interaction between Factor A and Factor B

How does the Test for main effect of Factor A map onto our null hypothesis?

H0: uA1 = uA2 = uA3

How does the Test for main effect of Factor B onto our null hypothesis?

H0: uB1 = uB

How does the Test for an interaction between Factor A and Factor B map onto our null hypothesis?

H0: there will be no interaction between factor A and factor B

How do we graph data from factorial designs?

Line graphs...?

How can we identify possible main effects from graphs?

If the lines are parallel, there can be a main effect

How can we identify possible interactions from graphs?

If the lines intersect at any point, there is an interaction

How does NHST for a factorial ANOVA differ from NHST for other designs that we've learned?

It has two IVs

What does a strong, thorough conclusion look like in NHST for factorial ANOVA?

"A ___ ANOVA revealed participants in their 3rd or 4th year of college spent significantly more hours studying (M=10.67, SD=1.21) than participants in their 1st or 2nd year in college (M=8.17, SD=1.17), F(1,8)=16.07, p=0.004, n2=0.57. The magnitude of this effect was large. However, this effect was qualified by a significant interaction with major (humanities vs. natural sciences), insert stats, such that the effect of year in college was stronger for humanities majors than natural science majors. There was no significant main effect of major on study time, F(1,8)=0.64, p=0.45, n2=0.02. The magnitude of this effect was small."

What effect size do we use for factorial ANOVA?

n^2

How do we interpret the effect size used in factorial ANOVA?

- Small = 0.01
- Medium = 0.06
- Large = 0.14

When is Tukey's HSD appropriate in a factorial design?

when we reject the null hypothesis