Factorial ANOVA Notes

Lecture Introduction

  • Apologies for the missing lecture recording from the previous Tuesday due to a hardware failure. A replacement is now in place.
  • The lecture from the previous year has been uploaded as a substitute, with similar content.
  • Content labeled as optional this year, such as understanding the sum of squares and degrees of freedom, and tests like correlation and chi-square tests, can be ignored if not covered this year.

Factorial ANOVA: Introduction

  • Today's focus: Factorial ANOVA, an advanced form of ANOVA for data grouped in multiple ways, expanding on the ANOVA family of tests.
  • Factorial ANOVA addresses scenarios with multiple grouping predictors, building upon the one-way ANOVA covered previously.
  • One-way ANOVA is similar to a t-test but allows for more than two groups, determining differences between the means of multiple groups with a continuous outcome measure and a categorical predictor.

ANOVA Terminology and Setup

  • Outcome Measure: ANIVA requires one continuous outcome measure; if there are multiple continuous outcomes, ANIVA is not suitable.
  • Categorical Predictor: Determine if there is one categorical predictor.
    • If not, ANIVA isn't appropriate.
    • If yes, determine the number of levels (groups).
  • Levels: The terms "group" and "level" are used interchangeably.
    • Two levels/groups: t-test.
    • Three or more levels/groups: One-way ANOVA.
  • Factors: Predictors are also referred to as factors, examining their influence and combined effects.
  • n-way Factorial ANOVA: If there are two predictors, it's a two-way factorial ANOVA.
    • One-way ANOVA always involves one predictor, regardless of the number of levels.
    • Factorial ANOVA involves more than one predictor.
    • The "two-way" prefix indicates the number of factors.
  • Example: Two factors, one with four levels, the other with two levels, is a "four by two" factorial ANOVA.
  • Worked Example: Today’s example is a two by three factorial ANOVA, with two predictors, one at two levels, one at three levels.
  • An ANOVA can have three, four, or five predictors (three-way, four-way, five-way factorial ANOVA), but the complexity increases significantly.
  • It is critical to identify the number and types of variables when approaching a research scenario to determine the appropriate statistical test.

Factorial ANOVA Procedure

  • Factorial ANOVA is similar to a one-way ANOVA in procedure but has a more extensive output due to testing more variables.

One-way ANOVA

  • Tests the main effect of a single factor (Factor A).
  • Determines if there is a difference between groups when grouped by this factor.
  • R output provides one row of results indicating a significant main effect of the predictor.

Two-way Factorial ANOVA

  • Tests multiple effects and provides answers for each.
    • Main effect of the first factor: Is there a difference in means when data is grouped by the first factor?
    • Main effect of the second factor: Is there a difference in means when data is grouped by the second factor?
    • Interaction between factors: Does one factor have a different effect depending on the level of the other factor?
  • An interaction occurs when one factor’s effect varies based on the level of the other factor.
  • Example: Caffeine's effect on test scores might differ based on sleep levels.
  • Factorial ANOVA systematically gives results for each of these questions.
  • Follow-up tests can be performed on the results.

Omnibus Test

  • An omnibus test assesses multiple things at once.
  • One-way ANOVA is already an omnibus test because it tests all the possible pairs of groups within a factor.
  • Factorial ANOVA is an even more aggregate omnibus test because it checks each factor effect and the interaction between them.

Independent Samples

  • Only independent samples are discussed, meaning different individuals are in each combination of groupings.
  • For example, in an experiment on sleep and caffeine, each subgroup has different individuals.
  • Versions exist for repeated measures, but these won't be covered.

Data Requirements

  • One continuous quantitative outcome variable.
  • Two categorical predictor variables with at least two levels each.

Research Questions Addressed

  • Are there differences between the means of different groups when grouped by either of the factors?
  • Is there an interaction between the factors?

F Statistics

  • The test yields three F statistics, each indicating the proportion of variability accounted for by each factor or interaction relative to within-group variation.
  • A larger F value indicates a stronger evidence of a difference between the groups or an interaction between the factors.
  • F statistics follow F distributions.

Research Scenario - Brain Training Apps

  • Investigating the effectiveness of brain training apps (e.g., Brainflex TM) on mental tasks, measured by the time to complete a Sudoku puzzle.
  • Outcome measure: Time in minutes to complete a Sudoku puzzle (continuous quantitative measure).
  • Experimental Design:
    • 40 participants receive Brainflex training.
    • 40 participants receive Sudoku training.
    • 40 participants form a control group (no training).
  • First Factor: Training, with three levels (Brainflex, Sudoku, Control).
    • Second Factor: Expertise, with two levels (Expert, Novice).
    • This setup is a three by two factorial ANOVA.

Hypotheses

  • Null Hypothesis: There is no main effect of either factor and there is no interaction between them or all subgroup means are equal.
  • Alternative Hypothesis: At least one mean is different.

Data Structure

  • Data includes columns for participant number, each predictor (with text labels for group), and the outcome (minutes to solve a Sudoku).
  • Data Visualization:
    • Factors are grouped along the x-axis, similar to a one-way ANOVA.
    • Expertise factor is visualized using color (e.g., blue for novices, red for experts).
    • Data spread suggests potential main effects or interactions.

Balanced Design

  • A balanced design has the same number of participants in each subgroup.
  • In the study, 20 participants in each subgroup (e.g., 20 experts, 20 novices in each training type).
  • ANOVA results are more reliable with balanced designs.

Assumptions

  • Factorial ANOVA has the same assumptions as one-way ANOVA.
    • Measurements are independent (different individuals).
    • The residuals are roughly normally distributed.
    • Variance is roughly equal across groups (homogeneity of variance).
  • Testing Assumptions:
    • Residuals are tested for normality using Shapiro-Wilk test.
    • Homogeneity of variance is tested using Levene's test.

Running and Interpreting Factorial ANOVA

  • Running the test is similar to a one-way ANOVA, but the output is more elaborate.
  • Output includes degrees of freedom, F values, and P values for each tested effect.
    • Main effect of Expertise
    • Main effect of Training
    • Interaction between Expertise and Training
  • Each row in the output corresponds to one of these effects, requiring interpretation.
  • In the example, all three effects (expertise, training, and their interaction) are significant based on the p-values.

Reporting the F Value

  • When reporting an F value, there are always two degrees of freedom.
    • One related to the number of subgroups.
    • The other related to the overall number of participants.
  • Example: "F(2, 114) = [F value], p = [p-value]"
  • Sum of squares and mean squares can be ignored and is optional.
  • Degrees of freedom determine the correct F distribution to use.

Interpreting Results

Main Effect of Training

  • Significant result indicates differences in Sudoku solving time across the three training groups.
  • Follow-up tests (pairwise comparisons) are needed to determine which groups differ significantly.

Main Effect of Expertise

  • Significant result indicates a difference in mean Sudoku solving time across the two expertise groups.
  • With only two groups, follow-up tests are unnecessary; the significant effect indicates the two groups are significantly different.
  • Numerically, novices are taking longer than experts to solve the puzzle.

Interaction Between Factors

  • Definition of Interaction: The effect of one factor differs depending on the level of the other factor.
  • In the example, if you had no training, expertise was really helpful. Experts ended up solving the sudoku quite a bit faster. Same with brain flex training, but with sudoku training the difference was much smaller.

Reporting Factorial ANOVA Results

  • Report the omnibus test results first.
  • State the name of the test: a two-way factorial ANOVA.
  • State the significance level (e.g., p < 0.05).
  • Report each row of the R output systematically.
  • Example statements:
    • "There was a main effect of prior expertise on Sudoku solving time, F(df, df) = value, p < 0.01."
    • "There was a main effect of training type on Sudoku solving time, F(df, df) = value, p = value."
    • "There was also an interaction between expertise and training type, F(df, df) = value, p=value."
  • Follow up with specific effects is optional if the researchers want more specific information.

Visualizing Interaction Effects

  • Visualizations are simplified to show dots representing group means.
  • Lines connect dots within the same group.

Scenario 1: Day of the Week vs. Caffeine Intake on Test Scores

  • X-axis: Day of the week (Monday, Tuesday).
  • Dot color: Caffeine intake (black = no caffeine, white = caffeine).
  • Outcome: Test score.
  • No main effect of day - the mean score on Monday was the same as the mean score on Tuesday if the factors are averaged.
  • Main effect of caffeine intake
  • No Interaction - The effect of caffeine doesn't depend on the day. If there was one, the effect of one factor would have a different effect being at a different level than the other factor.

Scenario 2: Food Item vs. Condiment on Enjoyment of Food

  • Y-axis: Enjoyment of food.
  • X-axis: Food item (hot dog, ice cream).
  • Dot color: Condiment (mustard, chocolate sauce).
  • No main effects.
  • Interaction is present - the effect of each condiment is dependent on what food it is on!

Patterns in Visualizations

  • When you have a sort of parallel shift like this, that tends to be an indication that you're probably looking at main effects.
  • Interactions tend to look like diagonals or crosses.

Example : 3x2 Factorial ANOVA

  • You have a quiz.
  • Activity: study/notes, coffee study
  • Timing:5 min, 1hr, 2 hours
  • Main Effect of activity - the mean test scores got different scores depending on when they were told to sit the test
  • Effect manifests as an interaction between these two things.
  • the three things that you get from any factorial ANOVA can occur in any combination and that whenever we run a factorial ANOVA we need to report all three of them regardless of whether they're significant or not.