Design of Experiments – Two-Way ANOVA & Post-Hoc Procedures

Two-Way ANOVA: Conceptual Overview

• Two (or more) independent categorical factors examined simultaneously → factorial design.
• Extends logic of one-way ANOVA; each unique combination of factor levels is a treatment/cell.
• Response values within a treatment are called replications.
• Primary questions addressed:
  • Main effect of Factor A (e.g., video-game violence).
  • Main effect of Factor B (e.g., participant sex).
  • Interaction effect A \times B: Does the effect of one factor depend on the level of the other?
• Hierarchy of testing: test interaction first; if significant, do not interpret main effects in isolation.

Illustrative Motivation

• Bartholow & Anderson (2002): video-game violence (violent vs non-violent) × sex (male vs female) → aggressive behaviour measured.
  • Main effect questions: Does violence raise aggression? Do males differ from females?
  • Interaction question: Do violence effects differ for males vs females?

Key Terminology & Notation

• a = # levels of Factor A.
• b = # levels of Factor B.
• r = # replications per treatment.
• ab = # treatments (cells).
• T{ij} = treatment total in cell (i,j), \bar y{ij} mean, s_{ij}^2 variance.
• T{\text{row} i} , T{\text{col} j} = row & column totals.
• Grand total G = \sum T{\text{row} i} = \sum T{\text{col} j}.
• Total # observations N = a\,b\,r.

Main-Effect Hypotheses

Factor A:

• H0: \mu{A1}=\mu{A2}=\dots=\mu{Aa}
• HA: at least two \mu{Ai} differ.
  Factor B: analogous.

Interaction Definition & Interpretation

• Definition 1: Interaction occurs when observed mean differences among cells are not what would be predicted from the additive main effects.
• Definition 2: One factor’s influence varies across levels of the other.
• Graphical cue: parallel cell-means lines → no interaction; non-parallel (cross, diverge, converge) → interaction present (interaction plot).
• Chemistry example: \text{HCl} and \text{NaOH} each caustic individually; combined they neutralise to \text{NaCl}+\text{H}_2\text{O} – illustrates non-additive effect.

ANOVA Decomposition

Stage 1 (identical to one-way):

• SS{\text{Total}} = SST + SS_E
• SST = \sum{k=1}^K nk (\bar yk - \bar y)^2 = \sum \frac{Tk^2}{nk} - \frac{G^2}{N}
• SSE = SS{\text{Total}} - SST = \sum (nk-1)sk^2 Stage 2: partition SST into
• SSA (between rows), SSB (between columns), SS{AB} (interaction): SSA = \sum \frac{T{\text{row} i}^2}{n{\text{row} i}} - \frac{G^2}{N}
  SSB = \sum \frac{T{\text{col} j}^2}{n{\text{col} j}} - \frac{G^2}{N} SS{AB} = SST - SSA - SS_B

Mean Squares & F-Ratios

• MSA = \dfrac{SSA}{a-1}
• MSB = \dfrac{SSB}{b-1}
• MS{AB} = \dfrac{SS{AB}}{(a-1)(b-1)}
• MSE = \dfrac{SSE}{ab(r-1)}
• Corresponding F tests:
  FA = \dfrac{MSA}{MSE}, FB = \dfrac{MSB}{MSE}, F{AB} = \dfrac{MS{AB}}{MS_E}
• Reject H0 if F > F{cv,\alpha}(\text{df}{num},\text{df}{den}); or p\text{-value}<\alpha.

Two-Way ANOVA Table Structure

| Source | SS | df | MS | F |
| Treatment (between) | SST | ab-1 | MST | MST/MSE |
| A | SSA | a-1 | MSA | FA | | B | SSB | b-1 | MSB | FB |
| A \times B | SS{AB} | (a-1)(b-1) | MS{AB} | F{AB} | | Error (within) | SSE | ab(r-1) | MSE | – | | Total | SS{\text{Total}} | N-1 | – | – |

Decision Guidelines

• Critical-value approach: if F_\text{ratio}>F{cv,\alpha} → reject H0.
• p-value approach: if p<\alpha → reject H_0.

Example 1 Summary

• Calculated F{AB}=10.42 with df=1,16; F{cv}=4.49 → significant interaction.
• Because interaction significant, main effects of factors A & B not tested independently.

Assumptions of Two-Way ANOVA

1. Normality within each treatment (check via side-by-side boxplots, etc.).
2. Homogeneity of variances across treatments.
3. Independence of observations.
4. Interval/ratio measurement scale.

Example 2: Direct-Mail Experiment

• Factors: Envelope (standard vs new logo) × Miles offer (none, double, anywhere) ⇒ 2×3 = 6 treatments; 5000 mailings each (total 30 000).
• ANOVA indicated:
  • Interaction p=0.61>0.05 → fail to reject interaction; lines roughly parallel in interaction plot.
  • Main effects significant for both Miles and Envelope (p<0.0001).

Post-Hoc (Multiple Comparison) Tests

• Purpose: identify which population means differ after significant overall ANOVA.
• Bonferroni procedure:
  1. Compute # pairwise comparisons: J = {ab \choose 2} = \dfrac{ab!}{2!(ab-2)!}.
  2. Standard error for mean difference \bar yi - \bar yj: s{\bar yi-\bar yj}=\sqrt{MSE \left(\tfrac{1}{ni}+\tfrac{1}{nj}\right)}.
  3. Confidence interval: \big(\bar yi-\bar yj\big) \pm t{\alpha^} s{\bar yi-\bar yj} where \alpha^ = \dfrac{\alpha}{2J} and df = ab(r-1).
  4. Interpret: interval including 0 ⇒ not significant; otherwise significant.

Learning Check Highlights

• Significant interaction does NOT imply anything definite about main-effect significance; their significance is independent (answer D).
• Misconceptions:
  • Two separate one-way ANOVAs cannot substitute for two-way ANOVA because they ignore interaction (statement A false).
  • Combining factors does not prevent assessment of each factor’s individual effect (statement B false).

Practical / Ethical / Philosophical Implications

• Interaction effects warn against simplistic additive thinking (e.g., combining caustic chemicals can neutralise, video-game violence impact may depend on sex).
• Proper hierarchical testing prevents misleading conclusions about main factors.
• Ensuring assumptions (normality, equal variances, independence) safeguards ethical integrity of inferential claims.