Apologies for the missing lecture recording from the previous Tuesday due to a hardware failure. A replacement is now in place.
The lecture from the previous year has been uploaded as a substitute, with similar content.
Content labeled as optional this year, such as understanding the sum of squares and degrees of freedom, and tests like correlation and chi-square tests, can be ignored if not covered this year.
Factorial ANOVA: Introduction
Today's focus: Factorial ANOVA, an advanced form of ANOVA for data grouped in multiple ways, expanding on the ANOVA family of tests.
Factorial ANOVA addresses scenarios with multiple grouping predictors, building upon the one-way ANOVA covered previously.
One-way ANOVA is similar to a t-test but allows for more than two groups, determining differences between the means of multiple groups with a continuous outcome measure and a categorical predictor.
ANOVA Terminology and Setup
Outcome Measure: ANIVA requires one continuous outcome measure; if there are multiple continuous outcomes, ANIVA is not suitable.
Categorical Predictor: Determine if there is one categorical predictor.
If not, ANIVA isn't appropriate.
If yes, determine the number of levels (groups).
Levels: The terms "group" and "level" are used interchangeably.
Two levels/groups: t-test.
Three or more levels/groups: One-way ANOVA.
Factors: Predictors are also referred to as factors, examining their influence and combined effects.
n-way Factorial ANOVA: If there are two predictors, it's a two-way factorial ANOVA.
One-way ANOVA always involves one predictor, regardless of the number of levels.
Factorial ANOVA involves more than one predictor.
The "two-way" prefix indicates the number of factors.
Example: Two factors, one with four levels, the other with two levels, is a "four by two" factorial ANOVA.
Worked Example: Today’s example is a two by three factorial ANOVA, with two predictors, one at two levels, one at three levels.
An ANOVA can have three, four, or five predictors (three-way, four-way, five-way factorial ANOVA), but the complexity increases significantly.
It is critical to identify the number and types of variables when approaching a research scenario to determine the appropriate statistical test.
Factorial ANOVA Procedure
Factorial ANOVA is similar to a one-way ANOVA in procedure but has a more extensive output due to testing more variables.
One-way ANOVA
Tests the main effect of a single factor (Factor A).
Determines if there is a difference between groups when grouped by this factor.
R output provides one row of results indicating a significant main effect of the predictor.
Two-way Factorial ANOVA
Tests multiple effects and provides answers for each.
Main effect of the first factor: Is there a difference in means when data is grouped by the first factor?
Main effect of the second factor: Is there a difference in means when data is grouped by the second factor?
Interaction between factors: Does one factor have a different effect depending on the level of the other factor?
An interaction occurs when one factor’s effect varies based on the level of the other factor.
Example: Caffeine's effect on test scores might differ based on sleep levels.
Factorial ANOVA systematically gives results for each of these questions.
Follow-up tests can be performed on the results.
Omnibus Test
An omnibus test assesses multiple things at once.
One-way ANOVA is already an omnibus test because it tests all the possible pairs of groups within a factor.
Factorial ANOVA is an even more aggregate omnibus test because it checks each factor effect and the interaction between them.
Independent Samples
Only independent samples are discussed, meaning different individuals are in each combination of groupings.
For example, in an experiment on sleep and caffeine, each subgroup has different individuals.
Versions exist for repeated measures, but these won't be covered.
Data Requirements
One continuous quantitative outcome variable.
Two categorical predictor variables with at least two levels each.
Research Questions Addressed
Are there differences between the means of different groups when grouped by either of the factors?
Is there an interaction between the factors?
F Statistics
The test yields three F statistics, each indicating the proportion of variability accounted for by each factor or interaction relative to within-group variation.
A larger F value indicates a stronger evidence of a difference between the groups or an interaction between the factors.
F statistics follow F distributions.
Research Scenario - Brain Training Apps
Investigating the effectiveness of brain training apps (e.g., Brainflex TM) on mental tasks, measured by the time to complete a Sudoku puzzle.
Outcome measure: Time in minutes to complete a Sudoku puzzle (continuous quantitative measure).
Experimental Design:
40 participants receive Brainflex training.
40 participants receive Sudoku training.
40 participants form a control group (no training).
First Factor: Training, with three levels (Brainflex, Sudoku, Control).
Second Factor: Expertise, with two levels (Expert, Novice).
This setup is a three by two factorial ANOVA.
Hypotheses
Null Hypothesis: There is no main effect of either factor and there is no interaction between them or all subgroup means are equal.
Alternative Hypothesis: At least one mean is different.
Data Structure
Data includes columns for participant number, each predictor (with text labels for group), and the outcome (minutes to solve a Sudoku).
Data Visualization:
Factors are grouped along the x-axis, similar to a one-way ANOVA.
Expertise factor is visualized using color (e.g., blue for novices, red for experts).
Data spread suggests potential main effects or interactions.
Balanced Design
A balanced design has the same number of participants in each subgroup.
In the study, 20 participants in each subgroup (e.g., 20 experts, 20 novices in each training type).
ANOVA results are more reliable with balanced designs.
Assumptions
Factorial ANOVA has the same assumptions as one-way ANOVA.
Measurements are independent (different individuals).
The residuals are roughly normally distributed.
Variance is roughly equal across groups (homogeneity of variance).
Testing Assumptions:
Residuals are tested for normality using Shapiro-Wilk test.
Homogeneity of variance is tested using Levene's test.
Running and Interpreting Factorial ANOVA
Running the test is similar to a one-way ANOVA, but the output is more elaborate.
Output includes degrees of freedom, F values, and P values for each tested effect.
Main effect of Expertise
Main effect of Training
Interaction between Expertise and Training
Each row in the output corresponds to one of these effects, requiring interpretation.
In the example, all three effects (expertise, training, and their interaction) are significant based on the p-values.
Reporting the F Value
When reporting an F value, there are always two degrees of freedom.
One related to the number of subgroups.
The other related to the overall number of participants.
Example: "F(2, 114) = [F value], p = [p-value]"
Sum of squares and mean squares can be ignored and is optional.
Degrees of freedom determine the correct F distribution to use.
Interpreting Results
Main Effect of Training
Significant result indicates differences in Sudoku solving time across the three training groups.
Follow-up tests (pairwise comparisons) are needed to determine which groups differ significantly.
Main Effect of Expertise
Significant result indicates a difference in mean Sudoku solving time across the two expertise groups.
With only two groups, follow-up tests are unnecessary; the significant effect indicates the two groups are significantly different.
Numerically, novices are taking longer than experts to solve the puzzle.
Interaction Between Factors
Definition of Interaction: The effect of one factor differs depending on the level of the other factor.
In the example, if you had no training, expertise was really helpful. Experts ended up solving the sudoku quite a bit faster. Same with brain flex training, but with sudoku training the difference was much smaller.
Reporting Factorial ANOVA Results
Report the omnibus test results first.
State the name of the test: a two-way factorial ANOVA.
State the significance level (e.g., p < 0.05).
Report each row of the R output systematically.
Example statements:
"There was a main effect of prior expertise on Sudoku solving time, F(df, df) = value, p < 0.01."
"There was a main effect of training type on Sudoku solving time, F(df, df) = value, p = value."
"There was also an interaction between expertise and training type, F(df, df) = value, p=value."
Follow up with specific effects is optional if the researchers want more specific information.
Visualizing Interaction Effects
Visualizations are simplified to show dots representing group means.
Lines connect dots within the same group.
Scenario 1: Day of the Week vs. Caffeine Intake on Test Scores
X-axis: Day of the week (Monday, Tuesday).
Dot color: Caffeine intake (black = no caffeine, white = caffeine).
Outcome: Test score.
No main effect of day - the mean score on Monday was the same as the mean score on Tuesday if the factors are averaged.
Main effect of caffeine intake
No Interaction - The effect of caffeine doesn't depend on the day. If there was one, the effect of one factor would have a different effect being at a different level than the other factor.
Scenario 2: Food Item vs. Condiment on Enjoyment of Food
Y-axis: Enjoyment of food.
X-axis: Food item (hot dog, ice cream).
Dot color: Condiment (mustard, chocolate sauce).
No main effects.
Interaction is present - the effect of each condiment is dependent on what food it is on!
Patterns in Visualizations
When you have a sort of parallel shift like this, that tends to be an indication that you're probably looking at main effects.
Interactions tend to look like diagonals or crosses.
Example : 3x2 Factorial ANOVA
You have a quiz.
Activity: study/notes, coffee study
Timing:5 min, 1hr, 2 hours
Main Effect of activity - the mean test scores got different scores depending on when they were told to sit the test
Effect manifests as an interaction between these two things.
the three things that you get from any factorial ANOVA can occur in any combination and that whenever we run a factorial ANOVA we need to report all three of them regardless of whether they're significant or not.