The analysis starts with evaluating the three F statistics, focusing primarily on the interaction effect.
If the interaction effect is significant, follow-up tests are conducted to interpret it. If not significant, the main effects are examined.
Evaluating F Statistics
For two factors (A and B):
Comparison begins by observing the F statistic for each factor.
A significant P-value (P < alpha) for any main effects indicates the need for post hoc tests, similar to those used in one-way ANOVA.
Post Hoc Tests
Conducted for each significant main effect identified from the ANOVA.
Example questions addressed in beverage testing:
Is the mean satisfaction of frappuccino greater than tea?
Are mean comparisons made for each beverage type individually?
For time of day, a similar analysis is performed to determine differences in satisfaction during different periods.
Stopping Criteria for ANOVA
If the interaction effect is not significant, no further post hoc tests are required for the factors.
Main effects should not be interpreted when interactions are significant, as they can obscure important differences.
Importance of Interaction Evaluation
Main effects provide average effects across levels of another factor, but interactions indicate that these effects can vary.
Misrepresentation can occur if the average is interpreted without acknowledging the interaction.
Example Scenario (Starbucks Satisfaction Study):
Average satisfaction of drinks can mislead conclusions if satisfaction changes across different times of the day.
Understanding Post Hoc Tests in Presence of Interaction
Post hoc tests used following significant ANOVA results must be relevant to the interactions.
Pairwise comparisons between groups still apply, with focus on controlling Type I error rates.
Review of two weeks ago's concepts:
Different post hoc methods balance assumptions, power, and conservativeness.
Types of Error Rates in Testing
Family-Wise Error Rate (FWER):
Controls probability of at least one false positive among all tests conducted.
Generally more conservative; it attempts to avoid any false discoveries.
False Discovery Rate (FDR):
Looks at the expected proportion of false positives in declarations.
Less conservative; can lead to higher power when many tests are conducted.
Review of Post Hoc Test Options
Grouped by assumptions of variance:
Equal Variances Assumed:
Bonferroni, Tukey, Scheffé (varies in conservativeness).
Unequal Variances:
Games-Howell, Dunnett’s C, Welch’s ANOVA.
Non-Parametric Options for Testing
Wilcoxon Rank-Sum Test:
Non-parametric alternative to traditional tests.
Can control Type 1 error rates within the scope of the non-parametric space.
Follow-Up Tests for Interaction Effects
Follow-up tests, termed simple effects, are necessary when determining the effect of one independent variable at various levels of another.
Example Analysis: Effects of drink type on satisfaction at different times of day.
Decomposing Interactions
Possible methods include:
Comparing drink type within specific time frames (e.g., morning, afternoon, evening).
Analyzing how different drinks perform within each time of day, using t-tests or ANOVA as suitable.
Trade-offs exist in terms of detail versus simplicity in analysis methods.
Simple Effects with Different Designs
For a 2x3 design:
Assess the two-level factor at each level of the three-level factor through multiple t-tests.
Coverage of group comparisons might include:
Conducting ANOVAs for individual paired variables, not exceeding the complexity of factorial designs.
ANOVA Testing Across Time and Beverage Types
Examining satisfaction within different time frames for each beverage.
Significance leads to follow-up tests determining whether post hoc tests are needed based only on significant interactions.
Visualization and Effect Size
The importance of plotting means to visualize differences in results.
Introduction to effect size measure (eta squared) to assess variability attributed to factors.
Eta squared ranges from 0 to 1, applicable for each factor and their interactions identified in the testing.
Conclusion and Further Questions
Emphasis on understanding and interpreting interactions effectively to avoid misrepresentation.
Questions from participants focus on specific testing scenarios and clarifications on methodologies, fostering a deeper understanding of the concepts discussed.