Understanding Post Hoc Comparisons
Introduction to Post Hoc Comparisons
Post hoc comparisons are analyses conducted after a significant ANOVA result to determine which specific groups differ from each other.
These comparisons can explore various hypotheses that were not explicitly planned in advance.
Distinction of comparisons:
Comparisons against the Control Group: Evaluate how each experimental group relates to the control.
Specific Comparisons: Select the groups to compare based on data trends or exploratory objectives.
All Possible Pairwise Comparisons: Analyze every combination of group comparisons.
Overview of Post Hoc Tests
Post Hoc Tests: Designed and conducted after data collection, focusing on controlling inflated family wise error rates during multiple comparisons.
Flexibility of Post Hoc Comparisons: Can range from broad (all possible comparisons) to specific (selective) comparisons.
Family Wise Error (FWE): Refers to the probability of making at least one Type I error across multiple comparisons.
Type I Error and Family Wise Error Rate
Type I Error (α): Occurs when the null hypothesis is incorrectly rejected when it is true.
Significance Level (α = 0.05): Indicates a 5% chance of making a Type I error per comparison.
Inflated Family Wise Error Rate: Involves increased risk of Type I error when making multiple comparisons, leading to a greater probability of incorrect conclusions.
Factors Influencing Family Wise Error
Number of Groups (k): More groups lead to more potential comparisons.
Number of Comparisons (c): Increases as group numbers increase, significantly impacting Type I error rates.
Example Calculation of Pairwise Comparisons
For 5 groups:
Total pairwise comparisons = 10 (calculated as (\binom{n}{2}=\frac{n(n-1)}{2}))
Explicit comparisons include:
Group 1 with Groups 2, 3, 4, 5 (4 comparisons)
Subsequent groups make remaining comparisons leading to a total of 10.
Visualization of Comparisons and Error Rates
As the number of groups increases, the number of comparisons grows rapidly:
Example: 18 groups result in 150 comparisons; thus, the likelihood of committing multiple Type I errors is high.
Family Wise Error Rate Formula and Adjustment
Family Wise Error Rate (FWER): Can be formulated as: FWER = 1 - (1 - \alpha^{\prime})^{c} Where (\alpha^{\prime} ) is the adjusted error rate per comparison.
The formula calculates the probability of making at least one Type I error across multiple comparisons.
Adjusting Alpha (α'): Adjusting the significance level can help control the inflated family wise error, increasing the likelihood of obtaining reliable results.
Comparison Strategies and Post Hoc Tests
Important to consider:
Which groups to compare (broad vs. selective comparisons).
How to manage inflated family wise error using appropriate statistical tests.
Types of Post Hoc Tests
Dunnett's Test:
Utilized when comparing a control group to several experimental groups.
Focuses solely on comparisons of each experimental group to the control group.
Formula Utilized: Incorporates parts of a t-test and involves calculating t-statistics based on differences between group means.
Formula generally is:
t = \frac{\overline{X}{c} - \overline{X}{j}}{\sqrt{\frac{MSE}{n}}}Where ( \overline{X}{c} ) = mean of control group, ( \overline{X}{j} ) = mean of experimental group, and ( MSE ) = mean squared error from ANOVA summary table.
Conclusion and Implications
Understanding post hoc tests and family wise error is crucial for interpreting ANOVA results correctly.
Post hoc tests provide pathways to explore significant differences between group means while mitigating the risk of Type I errors.
These methods allow researchers to make informed decisions based on statistically robust comparisons, enhancing the validity of their findings in experimental settings.