PSYC311: Research Design and Statistics III - Week 11 Lecture Notes

Mixed Design ANOVA

• Also known as Split-plot ANOVA.
• Involves at least one between-groups IV and one within-groups IV.
• DV: one continuous variable

Types of Mixed Design ANOVA

• 2-way mixed design ANOVA
  • One between-groups IV and one within-groups IV.
  • Example: Gender (male and female) and time of day (morning, afternoon, evening) on alertness.
• 3-way mixed design
  • Two between-groups IV and one within-groups IV.
  • One between-groups IV and two within-groups IV, etc.

Examples of IVs

• IV1 (between): Gender (Man vs Woman)
• IV2 (within): Treatment (pre, intervention, post)
• IV3 (between): School (NSW vs QLD)
  • 2 x 3 Mixed ANOVA (2-way)
  • 2 x 3 x 2 Mixed ANOVA (3-way)

ANOVA Shorthand

• The number of digits equals the number of variables.
• The number of the digit equals the number of levels.
• Indicate which variable is between-groups and which is within-groups.
• Example: Reaction time was compared using a 2(gender) x 3 (time of day) mixed design ANOVA, where gender was a between-groups factor and time of day was a within-groups factor.

Interpretation of Levels

• The number of levels of the within-groups IV indicates the number of times the DV was measured.

• The number of levels of the between-groups IV indicates the number of groups.

• Example: 2(gender) x 3 (time of day) mixed design ANOVA

  	Morning	Afternoon	Evening
  Gender
  Male	Participant 1-5	Participant 1-5	Participant 1-5
  Female	Participant 6-10	Participant 6-10	Participant 6-10

Information Obtained from a Two-Way Mixed Design ANOVA

• Main effects
  • Main effect for each IV.
  • Main effect for the within-groups factor on DV.
  • Main effect for between-groups factor on DV.
• Interaction
  • How the IVs interact with each other to influence the DV.

Assumptions

• Data being on an interval or ratio scale.
• Normal distribution.
  • Checking the distribution of the Between-groups IV.
  • Checking the distribution of the Within-groups IV.
• Homogeneity of variance.
  • For the Between-groups IV.
  • Check with Levene’s test for each measure of the DV (non-significant result is desired).
  • If violated: Between-groups Main effect and interaction are questionable.
• Sphericity.
  • For the Within-group factor.
  • Check with Mauchly’s test (non-significant result is desired).
  • Correct with Greenhouse-Geisser or Huynh-Feldt to both the within-groups main effect and the interaction.

Follow Up Main Effect

• Planned contrasts
  • When specific hypotheses have been made a priori.
• Post-hoc
  • For the within-groups main effect, Bonferroni is the safest bet.
  • For the Between-groups main effect:
    • When group sizes are equal: Tukey’s.
    • When group sizes are unequal: Scheffe (most conservative).

Describe Interaction

• Produce two graphs.
• Decide which one is easiest to describe.
• Write a description of what is in the graph.
• Describe the lines in the graph.
• Simple effects analysis
  • Analysis compares DV across the levels of one IV separately for the different levels of the other IV.
  • Use:
    • Within-group or Between-groups t-tests.
    • Within-group or between-groups ANOVA.

Jamovi: Entering Data

• Between groups Factor: one column for the IV (categorical).

• Within-group Factor: Each level of the IV is represented by a different column and the DV (for that condition) is entered into the column.

• 2(gender) x 3 (time of day) Mixed Design ANOVA

  Gender	Morning	Afternoon	Evening
  1	75	65	76
  2	65	73	65
  2	58	50	69
  2	73	78	58
  1	51	64	76
  1	70	45	65
  2	66	58	74

Jamovi Mixed Factorial Example

• In a speed dating scenario, male and female participants spoke to three attractive people of the opposite sex who varied in charisma: high, some, none.
• They were later asked to rate, on a 101-point scale how attractive they found the person (rating).
• Ratings were compared using a 2(gender: male, female) x 3 (charisma: high, some, none) mixed design ANOVA, where gender was a between-groups factor and charisma was a within-groups factor.

Check Normality

• Need to get it for all 6 conditions.
• Move the three levels of the within subjects factor over to variables.
• Split by the between subjects factor.

Check Normality Output

• Report includes skewness, kurtosis, and Shapiro-Wilk test results for each condition.
• If Shapiro-Wilk tests are non-significant, the assumption of normality is met.

Jamovi: Running Mixed Design ANOVA

• Analyses - ANOVA - Repeated Measures ANOVA.
• Move levels of within-subjects factor over.
• Move between-subjects factor over.
• Tell Jamovi what your levels of the within subjects factor is by typing them.

Check Assumptions in Jamovi

• Go to assumption checks and select:
  • Sphericity tests for your within subjects factor.
  • Homogeneity test for your between subjects factor.

Sphericity

• Mauchly’s test being non-significant indicates that the assumption of sphericity has been met.

Homogeneity of Variance

• Levene’s test is run for each level of the within-subjects IV to check homogeneity of variance.
• Non-significant Levene's tests indicate that you have met the assumption of homogeneity of variance.

Interpreting ANOVA Output

• The output is split over two different tables.
  • The within subjects effect contains the results for your within subjects IV and also your interaction.
  • The between subjects effects contains the between subjects IV.

Main Effect for Within Subjects IV - Charisma

• The main effect for charisma on rating was significant averaged across gender, F(2,36) = 64.8, p<.001, partial η^2 = .78.
• Follow up analysis required to get direction.

Post-hoc contrasts

• Estimated Marginal Means.
• Post Hoc Tests.

Post-hoc contrasts Interpretation

• No difference between attractive high charisma and attractive some charisma: t(18) = 0.55, p=1.0.
• Attractiveness was rated significantly higher in the high (t(18) = 10.43, p<.001) and average charisma (t(18) = 10.17, p<.001) conditions than in the low charisma condition averaged across gender.

Main Effect for Between Subjects IV - Gender

• The main effect for gender on rating was significant: F(1,18) = 82.7, p<.001, partial η^2 = .82.
• Need direction! No follow up analysis required because only 2 levels, but estimated marginal means required to get the averaged means to interpret.

Main Effect for Between Subjects IV - Gender Interpretation

• On average female ratings (M=88.03, SD=.92) were higher than male ratings (M=76.17, SD=.92).

Interaction

• The interaction between gender and charisma on rating was significant: F(2,36) = 57.6, p<.001, partial η^2 = .76.

Interaction – first interpretation

• There was no difference in rating between males and females in the high and some charisma conditions.
• However, in the no charisma condition the males had substantially lower ratings than the females.
• Independent samples t-tests are conducted to compare males and females for each charisma condition.

Interaction – first interpretation: t-tests

• There was no difference across males and females in ratings for the high and some charisma conditions: t(18) = -0.47, p=.644 and t(18) = 0.50, p=.625, respectively.
• However, there was a difference in rating across males and females in the no charisma condition: t(18) = 17.42, p<.001. Males had lower ratings than females in this condition.

Interaction – Second interpretation

• For females there was no difference in ratings across the three charisma conditions. Within-group ANOVA
• For males, however, while ratings did not differ across the high and some charisma conditions, they were substantially lower in the no charisma condition than the other two conditions. Within-group ANOVA

Interaction – Second interpretation: Split File

• Because we want to compare males and females separately we need to split the file by gender
• Create a filter

Creating Filters

• Filtering using the variable you are NOT comparing them on.
• Enter in the variable name you want to filter.
• Enter in group that you want the results to be isolated for inside ‘ ‘.
• Need to put ==.
• The use of capitals/no capitals is VERY IMPORTANT

Interaction – Second interpretation – FEMALES

• For females, ratings did not differ across the three charisma conditions: F(2,18)=0.11, p=.896.

Interaction – Second interpretation – MALES

• For males however, ratings did differ across the three charisma conditions: F(2,18)=125, p<.001.

Post-hoc tests Males

• Post-hoc analysis revealed that for males, while ratings did not differ across the high and some charisma conditions, the ratings in the no charisma condition significantly differed to those in the other two conditions.
• As can be seen in the table no charisma had significantly lower ratings than both high and some charisma conditions.

Interaction – Second interpretation

• Post-hoc analysis revealed that for females, while ratings did not differ across the high and some charisma conditions, the ratings in the no charisma condition significantly differed to those in the other two conditions.
• As can be seen in the table no charisma had significantly lower ratings than both high and some charisma conditions.

Pulling it all together: The write up

• Figure 1 presents the average ratings for males and females across the three charisma conditions. As can be seen in the figure, there was very little difference in the ratings across the conditions, except the ratings for males in the no charisma condition were lower than the ratings in the other conditions.
• Must present the descriptive for all conditions and groups.

The write up

• Ratings were compared using a 2(gender: male, female) x 3 (charisma: high, some, none) mixed design ANOVA, where gender was a between-groups factor and charisma was a within-groups factor.
• The main effect for charisma on rating was significant averaged across gender, F(2,36) = 64.82, p<.001, partial η^2 = .78. Planned contrasts revealed that there was no difference in rating across the high (M=88.95, SD=6.06) and some charisma (M=87.80, SD=6.17) conditions: F(1,18) = 0.31, p=.586, partial η^2 = .02. However, the ratings in the no charisma condition (M=69.55, SD=18.73) were significantly lower than those for some charisma condition: F(1,18) = 103.57, p<.001, partial η^2 = .85.

The write up

• The main effect for gender on rating averaged across charisma level was significant: F(1,18) = 82.7, p<.001, partial η^2 = .82. On average ratings for females (M=88.03, SD=.92) were higher than those for males (M=76.17, SD=.92).
• The interaction between gender and charisma on rating was significant: F(2,36) = 57.6, p<.001, partial η^2 = .76. There was no difference across males and females in ratings for the high and some charisma conditions: t(18) = 0.47, p=.644 and t(18) = 0.50, p=.625, respectively.
• However, there was a difference in rating across males and females in the no charisma condition: t(18) = 17.4, p<.001. As can be seen in Table1/Figure1, males had higher rating than females in this condition.