Study Notes on Multiple Comparisons in Statistics

Overview of Multiple Comparisons in Statistics

Introduction to Multiple Comparisons

  • Transition from ANOVA (Analysis of Variance) to Multiple Comparisons.

  • Definition of multiple comparisons and their significance in statistical analysis.

Understanding ANOVA

  • ANOVA assesses total variability and divides it into two main components:

    • Between Groups Variability:

    • Represents deviation of each group mean from the grand mean.

    • Within Groups Variability:

    • Represents deviations between individual scores and their respective group means.

Calculating ANOVA Statistic

  • Calculation involves:

    • Working out sums of squares: Amount of squared deviations around the mean.

    • Dividing by degrees of freedom to derive the statistic for one-way ANOVA.

  • F-Ratio Definition:

    • F = rac{MS_{between}}{MS_{within}}

    • Where MS_{between} is the mean squares for between groups, and MS_{within} is the mean squares for within groups.

Importance of Mean Squares in Multiple Comparisons

  • Mean squares within is frequently used in multiple comparisons because:

    • It measures random error, representing total random variability in the dataset.

    • Used to determine if the observed differences between means exceed random error in the data.

Omnibus Nature of ANOVA

  • ANOVA is an omnibus test, which means:

    • It assesses all group means simultaneously to test the null hypothesis (that all means are equal).

  • If the F statistic is significant (usually if p < 0.05):

    • Null hypothesis can be rejected,

    • Concludes at least two treatment means differ significantly.

Types of Multiple Comparisons Approaches

  • Multiple comparisons depend heavily on experimental design:

    • Design determines how the independent variable is manipulated and how groups relate to each other.

Types of Hypotheses and Comparisons

  1. Planned Comparisons:

    • Specific, pre-determined, and focused hypotheses set up before the data collection.

    • Aim to compare a control group against experimental groups efficiently.

  2. Post Hoc Comparisons:

    • Conducted after data is collected;

    • Offers flexibility based on data outcomes without prior planning.

  3. Broad Approach:

    • Examines all possible pairwise comparisons in the dataset.

Distinctions between Planned and Post Hoc Comparisons

  • Planned Comparisons:

    • Require adherence to pre-established hypotheses.

    • Considered inflexible since they cannot change irrespective of emerging data patterns.

    • Smaller in quantity since they focus only on key hypotheses.

  • Post Hoc Comparisons:

    • Flexible and adaptable to findings from data post-collection.

    • Allow for a broader spectrum of comparisons, potentially including all groups.

    • Address inflated family-wise error rates through specific correction methods.

Types of Post Hoc Comparisons

  • Analysis of what comparisons are made and methods for correcting family-wise error rates:

    • Three Types of Post Hoc Comparisons:

    • Focus on both comparisons being made and family-wise error correction.

  • Emphasis on time scale:

    • Planned comparisons occur during the planning phase, while post hoc analyses relate directly to findings from the data.

Visual Representation of Planning and Analysis Phases

  • The graphic would illustrate development and planning stages necessary for comparisons, distinguishing when comparisons should be made in relation to data collection.