RSM 3/10/26

Significance Testing in ANOVA

  • The F-test is used to determine if there are significant differences among group means.
    • It is conducted only if the F-test is significant, which is defined by a p-value lower than a chosen threshold (common thresholds include 0.05 or 0.1).

Post Hoc Comparison Techniques

  • Pairwise Comparison Techniques

    • Least Significant Difference (LSD)

    • Used when the F-test is significant.

    • Calculates a single value to determine if the difference between two means is significant.

    • Bonferroni Method

    • A more conservative method than LSD.

    • Requires a larger difference between means to declare them as significantly different.

    • General Characteristics of Methods:

    • Techniques can be categorized as conservative (e.g., Bonferroni) or liberal (e.g., LSD).

      • A conservative approach requires larger differences for significance.
      • A liberal approach allows for smaller differences to be significant.

  • Assumptions for Method Selection

    • The choice between different separations depends on:
    • The type and nature of data.
    • Acceptance criteria of the journals in which findings are to be published.

  • Common Techniques:

    • LS means (Least Squares means): similar usage to LSD but potentially with different assumptions;

    • PD (Pairwise Differences): also used similarly to LSD for comparative analyses;

    • Multiple Range Tests:

    • Newman-Keuls Test: Uses a structured approach for comparing means, offering flexibility based on treatment arrangements.

      • For example, if there are four treatments, you compute comparisons resulting in three outcomes (computed as treatment count - 1).

    • Turkey's HSD (Honestly Significant Difference) and Student's Neuman-Keuls (SNK) Test: additional methods under multiple range categorizations.

Pairwise vs. Multiple Range Comparisons

  • Pairwise Comparisons:

    • Allow for pre-planned comparisons
    • Example Combinations:
    • Treatment 1 vs. Treatment 2 vs. Treatment 3 vs. Treatment 4.
      • Example questions: Is Treatment 1 and 2 significantly different from Treatment 3 and 4?
      • Combinations structured as: Treatment A vs. Treatment B vs. etc.

  • Degrees of Freedom in Comparisons:

    • Each contrast uses one degree of freedom, thus limiting the number of contrasts.
    • Example: With three treatments, only two contrasts can be made (computed as number of treatments - 1).

Example Experimental Design

  • Discussion of a two by two factorial experiment:
    • Two sources of grain: Barley and Corn.
    • Two processing methods: Coarse and Fine.
    • This results in four total treatment conditions.
    • Degrees of freedom thus calculated provides insight into how many contrasts can be executed:
    • Given four treatments, you can perform three contrasts (comparing grain source and processing methods).

Interaction Effects

  • Definition of Interaction:
    • The effect of one treatment varies depending on the level of another treatment factor.
    • Example Observations:
    • With barley, processing finer yields higher intake whereas with corn, finer processing can lead to lower energy gains.

Coefficients in Designed Experiments

  • To calculate contrasts, coefficients are formulated such that the total sums equal zero, e.g., comparing barley to corn:

    • Coefficients for

    • Barley vs. Corn: 1 (Barley), -1 (Corn); Total = 0.

    • General linear and quadratic contrasts can be determined from treatment levels in experimentation settings (linear: -1, 0, +1; quadratic: +1, -2, +1).

Gradient Treatment Designs

  • Distance between treatments must be consistent for valid linear and quadratic contrast coefficients.
  • If treatments are unequally spaced, alternative procedures (e.g., IML in SAS) are required to determine coefficients properly.

Practical Considerations in Statistical Tests

  • As more treatments are included in an analysis, the chance of type I error increases.
  • Important to follow experimentation design and selection of statistical techniques carefully to avoid inflated errors in interpretation.

Conclusion and Application of Tests

  • Practical application of techniques such as LSD in determining significant differences involves calculations based on degrees of freedom, mean separations, and statistical values derived from distribution tables.
  • Example from analysis: If comparing mean differences involves significant contrasts based on calculated thresholds, the treatment differences are concluded as significant.