L10_factorial_post_hoc

Page 1: Post Hoc Analysis of Factorial ANOVA

• Understanding post hoc tests is crucial after performing factorial ANOVA, as they help to analyze differences between group means after establishing significant interactions.

Page 2: Example Data

Groupings:

• Men and Women: Young, Middle, Older

• Inattention Scores:

  • Men: 12, 8, 3, 5, 3, 8

  • Women: 14, 6, 2, 2, 8, 5

Summary Statistics:

• Mean scores (𝑥𝑖) calculated for all groups

• Standard deviations (si) calculated

Page 3: Hypotheses for ANOVA Tests

Main Effects:

1. Gender:

   • H0: 𝜇G1 = 𝜇G2

   • HA: H0 is not true

2. Age:

   • H0: 𝜇A1 = 𝜇A2 = 𝜇A3

   • HA: H0 is not true

Interaction Effects:

3. Interaction:

   • H0: No interaction exists

   • HA: H0 is not true

Page 4: ANOVA Results Summary

Source Data:

Page 5: Figures of Inattention Scores

• Graphs displaying inattention scores across age groups for both men and women.

Page 6: Conducting Post Hoc Tests

Conditions for Post Hoc on Main Effects:

• Main effect must be significant

• Main effect should have three or more levels

• Interaction must not be significant

Page 7: Conducting Post Hoc Tests on Interaction

• If interaction is significant, do not conduct post hoc on main effects

• Instead, focus on cell means to interpret interaction patterns

Page 8: Gender Main Effect Findings

• Significant main effect found, F(1,42) = 6.86, ηp² = 0.14, p < .05.

• Males: 6.75, Females: 4.96

• No post hoc test on gender due to only 2 levels.

Page 9: Age Effect Findings

• Significant main effect of age, F(2,42) = 29.32, ηp² = 0.58, p < .01.

• Age Means: Young = 9.44, Middle = 4.88, Old = 3.25

• No post hoc test due to significant interaction.

Page 10: Gender X Age Interaction Findings

• Significant interaction, F(2,42) = 18.72, ηp² = 0.47, p < .01.

• Tukey’s HSD post hoc test conducted on cell means following the interaction.

Page 11: Graphical Representation of Interaction

• Showing interaction among genders and age groups regarding inattention scores.

Page 12: Tukey’s HSD Calculation

Components:

• q from Tukey’s table

• Degrees of freedom (df) and number of means (k): 6

• q value derived: 4.23

Page 13: ANOVA Table

• Detailed breakdown of ANOVA results previously summarized.

Page 14: Tukey Test Comparisons

Significant Differences:

• Significant differences found if means differ by more than 3.54 at α = .05

Comparison Groups Defined:

• Include males, females, and age groups for analysis.

Page 15: Age Comparisons in Males

Significant Findings:

• Young males differ from middle and older, (no difference in older and middle).

Page 16: Age Comparisons in Females

• No significant differences among age groups.

Page 17: Gender Comparisons Across Age Groups

Significant Findings:

• Young males differ significantly from young females.

• No differences for middle and older males vs females.

Page 18: Summary of Key Findings

• Significant main effects of gender and age

• Significant interaction identifying greater inattention among young males over females and older ages.

Page 19: A Second Example

Design Introduction:

• 2 (test format) x 3 (reading ability) between-subjects design.

• Reading ability: low, medium, high.

Page 20: Test Score Display

• Graph displayed showing results based on reading ability for both written and typed tests.

Page 21: ANOVA Results for Second Example

Page 22: Reading Ability Findings

• Main effect significant, F(2,12) = 4.41, ηp² = 0.42, p < .05.

Page 23: Test Format Findings

• Main effect not significant, F(1,12) = 0.12, ηp² = 0.01, p > .70.

Page 24: Interaction Findings

• Significant interaction detected, F(2,12) = 5.20, ηp² = 0.46, p < .05.

Page 25: Summary ANOVA Table

• Summarized results similar to Page 21.

Page 26: Tukey’s HSD Computation

Criteria Used:

• Analysis showed any means differing by more than 9.48 at α = .05 indicates significant differences.

Page 27: Results for Typed Test

• No significant differences in scores among students classified into low, medium, high reading abilities.

Page 28: Results for Written Test

Findings:

• Medium reading ability significantly higher than high reading ability.

Page 29: Typed vs. Written Test Differences

• Summary of differences noted between test formats at various reading ability levels.

Page 30: Summary of Main Effects

• Significant main effect of reading ability and test format noted, with interaction effects analyzed.

Page 31: Third Example Introduction

• Design details for another study focusing on test format and reading.

Page 32: Test Score Display for Third Example

• Displaying scores based on reading ability for written versus typed formats.

Page 33: ANOVA Table for Third Example

• Detailed results including significant main effects from reading ability and test format.

Page 34: Reading Ability Findings

• Significant main effect, with numeric results and indication for the need for post hoc testing.

Page 35: Test Format Findings

• Detailed results showing significant findings on the effect of test format on performance.

Page 36: Interaction Findings in Third Example

• Interaction findings did not reach significance, hence no additional analysis.

Page 37: Summary of Significant Statistics

• Detailed ANOVA findings for readability and understanding; values indicate significance.

Page 38: Tukey’s HSD Computation in Reading Ability

• Highlights the criteria used for analysis and the implications of mean differences.

Page 39: Closing Summary for Reading Ability

• Significant differences across comparisons, demonstrated through statistical output.

Page 40: Overall Summary of Findings

• Highlight of significant main effects for reading ability and test format, interaction not significant.

Page 41: Language Precautions

Proper Terminology:

• Use precise language when discussing hypotheses and statistical results to avoid misinterpretations.

Page 42: Significance Interpretations

Key Pointers:

• Significant results are binary; no degrees of significance (e.g. highly significant) as they either are or are not significant.

Post Hoc Analysis of Factorial ANOVA

Understanding post hoc tests is crucial after performing factorial ANOVA as they help analyze differences between group means after establishing significant interactions. These tests allow researchers to determine which specific groups are different from each other, providing deeper insights into the data collected.

Example Data

Groupings:

• Men and Women: Young, Middle, OlderInattention Scores:

• Men: 12, 8, 3, 5, 3, 8

• Women: 14, 6, 2, 2, 8, 5Summary Statistics:

• Mean scores (𝑥𝑖) calculated for all groups, indicating the average inattention scores among the different genders and age groups.

• Standard deviations (si) calculated to assess the dispersion of inattention scores within each group.

Hypotheses for ANOVA Tests

Main Effects:

• Gender:

  • H0: 𝜇G1 = 𝜇G2 (Null Hypothesis: No difference in means between genders)

  • HA: H0 is not true (Alternative Hypothesis: There is a difference)

• Age:

  • H0: 𝜇A1 = 𝜇A2 = 𝜇A3 (Null Hypothesis: No difference in means across ages)

  • HA: H0 is not true

Interaction Effects:

• Interaction:

  • H0: No interaction exists (The effect of one factor does not depend on the level of the other factor)

  • HA: H0 is not true

ANOVA Results Summary

Source Data:

This ANOVA results summary provides a clear breakdown of how each effect contributes to the overall variability in inattention scores.

Figures of Inattention Scores

Graphs displaying inattention scores across age groups for both men and women visually illustrate the trends and significant differences indicated in the statistical tests.

Conducting Post Hoc Tests

Conditions for Post Hoc on Main Effects:

• Main effect must be significant (p < .05)

• Main effect should have three or more levels (ensures enough comparisons can be made)

• Interaction must not be significant (to isolate the effect of each main factor)

Conducting Post Hoc Tests on Interaction

If interaction is significant, do not conduct post hoc on main effects. Instead, focus on cell means to interpret interaction patterns, as this provides better insights into how variables affect outcomes collectively rather than independently.

Gender Main Effect Findings

Significant main effect found, F(1,42) = 6.86, ηp² = 0.14, p < .05.

• Males: Average score = 6.75

• Females: Average score = 4.96

• No post hoc test on gender due to having only 2 levels, limiting comparison options.

Age Effect Findings

Significant main effect of age, F(2,42) = 29.32, ηp² = 0.58, p < .01.

• Age Means: Young = 9.44, Middle = 4.88, Old = 3.25

• No post hoc test due to significant interaction detected, which suggests that interactions between age and gender significantly affect inattention scores.

Gender X Age Interaction Findings

Significant interaction observed, F(2,42) = 18.72, ηp² = 0.47, p < .01, indicating that the relationship between age and inattention scores differs by gender.

• Tukey’s HSD post hoc test conducted on cell means following the interaction allows for detailed analysis of these variations.

Graphical Representation of Interaction

Graphs depict interaction among genders and age groups regarding inattention scores, clarifying how scores differ significantly across combinations of gender and age groups.

Tukey’s HSD Calculation

Components:

• q from Tukey’s table and degrees of freedom (df) and number of means (k): 6

• q value derived: 4.23, which is used to assess significant differences between group means in post hoc analysis.

ANOVA Table

Detailed breakdown of ANOVA results provides indispensable information on how mean differences contribute to overall analysis significantly.

Tukey Test Comparisons

Significant Differences:

• Significant differences found if means differ by more than 3.54 at α = .05, highlighting which specific group comparisons led to statistical significance.Comparison Groups Defined:

• Include males, females, and age groups for a comprehensive analysis of inattention scores.

Age Comparisons in Males

Significant Findings:

• Young males differ significantly from middle and older, indicating that age significantly affects inattention in males (no difference in older and middle age groups).

Age Comparisons in Females

• No significant differences noted among age groups, suggesting that female inattention scores are relatively consistent across age ranges.

Gender Comparisons Across Age Groups

Significant Findings:

• Young males differ significantly from young females, indicating a notable disparity in inattention.

• No differences for middle and older males in comparison to females, showing consistency in inattention scores.

Summary of Key Findings

• Significant main effects of gender and age indicate how factors influence inattention.

• Significant interaction shows heightened inattention among young males compared to females and older age groups, underscoring the importance of focusing on demographic interactions in analyses.

A Second Example

Design Introduction:

• 2 (test format) x 3 (reading ability) between-subjects design, where each participant is categorized based on their reading ability: low, medium, high, affecting test outcomes.

Test Score Display

Graph illustrates results based on reading ability for both written and typed tests, providing a visual representation of performance differences.

ANOVA Results for Second Example

Source:

Reading Ability Findings

Main effect significant, F(2,12) = 4.41, ηp² = 0.42, p < .05, confirming that reading ability affects test performance significantly.

Test Format Findings

Main effect not significant, F(1,12) = 0.12, ηp² = 0.01, p > .70, suggesting that the format of the test does not significantly impact performance across the sample.

Interaction Findings

Significant interaction detected, F(2,12) = 5.20, ηp² = 0.46, p < .05 indicates that the combination of reading ability and test format influences outcomes, necessitating a more detailed analysis.

Summary ANOVA Table

Summarized results similar to Page 21, providing necessary metrics for understanding group outcomes.

Tukey’s HSD Computation

Criteria Used:Analysis showed any means differing by more than 9.48 at α = .05 indicates significant differences across reading abilities in terms of test performance.

Results for Typed Test

No significant differences in scores among students classified into low, medium, high reading abilities, suggesting that performance across these classifications remains consistent in a typed format.

Results for Written Test

Findings:Medium reading ability significantly higher than high reading ability, indicating that those with medium reading skills perform better than those with high skills in a written test format.

Typed vs. Written Test Differences

Summarized differences noted between test formats at various reading ability levels, affecting conclusions drawn from performance assessments.

Summary of Main Effects

Significant main effect of reading ability and test format noted, while interaction effects analyzed to better understand underlying trends in data.

Third Example Introduction

Design details for another study emphasizing the interplay between test format and reading ability in influencing performance outcomes among participants.

Test Score Display for Third Example

Displaying scores based on reading ability, emphasizing how participants fared differently across written versus typed formats.

ANOVA Table for Third Example

Detailed results including significant main effects from reading ability and test format are outlined for comprehensive understanding.

Reading Ability Findings

Significant main effect; numeric results highlight the need for post hoc testing to understand pairwise comparisons further.

Test Format Findings

Detailed results indicate significant findings on the effect of test format on performance, clarifying how format influences student achievement and test outcomes.

Interaction Findings in Third Example

Interaction findings did not reach significance, hence no additional analysis is warranted, reflecting the need for robust methodology and careful interpretation of results.

Summary of Significant Statistics

Detailed ANOVA findings for readability and understanding; values indicate significance and the importance of statistical rigor in deriving conclusions from research data.

Tukey’s HSD Computation in Reading Ability

Highlights the criteria used for analysis and implications entailed from observed mean differences.

Closing Summary for Reading Ability

Significant differences across comparisons demonstrated through statistical output, revealing the importance of analytical techniques in interpreting educational assessment data.

Overall Summary of Findings

Highlight of significant main effects for reading ability and test format, specifying that while interactions were not significant, the distinct impacts of each variable remained clear and necessitated further investigation.

Language Precautions

Proper Terminology:Use precise language when discussing hypotheses and statistical results to avoid misinterpretations, ensuring clarity in communication of research findings.

Significance Interpretations

Key Pointers:Significant results are binary; no degrees of significance (e.g., highly significant) as they either are or are not significant, ensuring accurate representation of data outcomes in research reporting.