Factorial ANOVA Comprehensive Study Guide

Revision of One-Way ANOVA Analysis

• Foundations of One-Way ANOVA:     * Factorial ANOVA is built heavily upon the basis of one-way ANOVA analysis.     * The Stata command for a one-way ANOVA is generally one-way DV IV or anova DV IV.     * Variables:         * Dependent Variable (DV): Must be a numerical variable.         * Independent Variable (IV): A categorical variable containing different groups or levels.     * Determining Group Count: The number of groups in an IV can be determined by looking at the Between Groups Degrees of Freedom ($DF_{between}$). The formula is: $DF = k - 1$. For example, if $DF = 2$, the number of groups ($k$) is $3$.

• Research Questions and Hypotheses:     * Goal: To determine if there are between-group differences on the DV.     * Omnibus ANOVA: The first step of analysis which tests all groups simultaneously to see if any group exhibits a difference on the DV.     * Statistical Null Hypothesis ($H_0$): There isn't any difference at all among all the groups.

• Test Statistics and Effect Size:     * F-Value: Represents a ratio of "Signal to Noise." It is the ratio of between-group differences to within-group differences.     * Interpretation: A large F-value combined with a small P-value (e.g., $P < 0.05$) indicates statistically significant differences. If the F-value is close to $1$, it is unlikely that between-group differences will be observed.     * Effect Size ($\eta^2$): Calculated as the Sum of Squares (Between) divided by the Sum of Squares (Total). $\text{Eta Square} = \frac{SS_{between}}{SS_{total}}$.     * Example: In a sample calculation provided, $74.5 / 2081$ resulted in a very small effect size.

• Secondary Analysis (Follow-up Analysis):     * Purpose: While the omnibus test indicates that a difference exists, secondary analysis is required to identify which specific groups differ.     * Control Methods: Vital to control the Type I Error Rate inflation.     * Post-hoc Comparisons (e.g., Tukey): Used for pairwise comparisons of every possible group pair. Tukey's method is noted for providing the most power in post-hoc contexts.     * Planned Comparisons: An alternative approach where specific comparisons are signaled using customized contrast coefficients or shortcut commands.     * Bonferroni Adjustment: Another method used, particularly in planned comparisons, to control error rates.     * Effect Size for Pairs: Cohen's $d$ ($d = \frac{\text{Contrast}}{\text{Square root of Mean Square Error}}$).

Introduction to Factorial ANOVA

• Definition: Factorial ANOVA involves having more than one independent variable (factor) within a dataset. We focus primarily on designs with two independent variables.

• Core Characteristics:     * Numerical DV: There is still one numerical dependent variable.     * Categorical IVs: Multiple categorical independent variables.     * Levels: Each IV can include two or more levels.     * Design Type: Focuses on Between-Subject Design, where groups/cells are independent of each other.     * Experimental Nature: Designs can be experimental or quasi-experimental. Causal relationships can be inferred if the researcher has control over how groups are formed.

• Orthogonality and Fully Crossed Factors:     * The independent variables themselves should be independent (orthogonal) of each other.     * Fully Crossed: Levels within each IV must be combined to create a matrix of every possible combined condition.

• Examples of Crossed Designs:     * Costume and Age:         * IV 1 (Costume): Mickey, Superman, Batman.         * IV 2 (Age): $2$ to $4$ years old, $5$ to $8$ years old.         * Result: A $2 \times 3$ matrix yielding $6$ combined conditions (e.g., $2$-$4$ year olds wearing Mickey, $5$-$8$ year olds wearing Superman).     * Therapy and Severity:         * IV 1 (Therapy): ACT, CBT, Medication, Control group.         * IV 2 (Severity): Mild depression, Moderate depression.         * Result: A $2 \times 4$ matrix yielding $8$ subgroups (cells).

The Modulating Effect (Statistical Interaction)

• Definition: The effect of one independent variable depends on the levels of the other independent variable. This is distinct from "Statistical Control" found in multiple regression models.

• Forms of Interaction:     1. The effect is present in one level of the second IV but absent in another.     2. The presence of one IV changes the direction or magnitude of the other IV's effect on the DV.     3. The effect size is unequal across different levels of the other IV.

• Illustrative Scenarios:     * Pet Emotional Reaction: Investigating how Mood (Happy, Sad, Angry, Excited) affects Dogs vs. Cats. The effect of mood might be evident in dogs but absent in cats.     * Food and Condiments: Investigating if hot dogs vs. ice cream (IV 1) and mustard vs. chocolate sauce (IV 2) affect joy (DV). Mustard increases joy for hot dogs but decreases it for ice cream (Direction reversal).     * Pain in Athletes: Comparing levels of pain (DV) after a $10$k race vs. a full marathon (IV 1) for athletes vs. non-athletes (IV 2). The pain difference between a marathon and a $10$k is likely much smaller for athletes than for non-athletes.

Factorial Terminology and Research Questions

• Naming Conventions:     * Factor: Another term for an Independent Variable.     * Levels: Groups or conditions within a factor.     * Design Notation:         * X-way design: Named by the number of factors (e.g., $2$-way ANOVA has $2$ factors).         * A by B design: Written as $X \times Y$ where $X$ and $Y$ represent the number of levels in each factor (e.g., a $2 \times 2$ design).

• Effects in a 2-way Factorial Design:     1. Main Effect of Factor A: Does Factor A affect the DV, disregarding Factor B? (Uses marginal means).     2. Main Effect of Factor B: Does Factor B affect the DV, disregarding Factor A? (Uses marginal means).     3. Interaction Effect (A x B): Does Factor B modulate the effect of Factor A on the DV? Or, is the effect of Factor A different at different levels of Factor B?     * Calculating this involves comparing the differences between means.

• Complexity of 3-way Designs:     * A $3$-way design produces $7$ total F-values: $3$ main effects, $3$ two-way interactions (A x B, B x C, A x C), and $1$ three-way interaction (A x B x C).

Data Summary: Means and Graphs

• Descriptive Table Components:     * Cell Means: Means associated with a specific combined subgroup (e.g., Male Freshman confidence level).     * Marginal Means: The mean for a single factor level while ignoring the other factor (appearing on the margins of the table).     * Grand Mean: The average of the entire participant cohort.

• Visualization Types:     * Line Graph: Most common for seeing interaction. Points represent cell means. Interaction is suggested if lines are non-parallel or crossing.     * CI Plot (Confidence Interval Plot): Shows means and error whiskers.     * Bar Graph: The height of the bar represents the cell mean.     * Plotting Rule of Thumb: Put the factor with more levels on the x-axis and the factor with fewer levels as separate colored lines/bars.

Statistical Concepts and Variance Partitioning

• Total Variability ($SS_{Total}$): The overall variance in the DV.

• Decomposition of Variance:     * One-Way ANOVA: Total variance = Between-group variance + Within-group variance (Error).     * Factorial ANOVA: Further breaks down between-group variance:         * $SS_{Total} = SS_{Model} + SS_{Residual}$         * $SS_{Model} = SS_{Factor A} + SS_{Factor B} + SS_{Interaction (AB)}$

• Degrees of Freedom (DF):     * $DF_{Factor A} = \text{Levels of A} - 1$     * $DF_{Factor B} = \text{Levels of B} - 1$     * $DF_{Interaction} = DF_A \times DF_B$     * $DF_{Total} = N - 1$ ($N$ = entire sample size).     * $DF_{Residual} = DF_{Total} - DF_{Model}$.

• Effect Size - Partial Eta Square ($\eta^2_p$):     * Used instead of regular eta square in factorial designs to allow comparisons across studies regardless of how many factors are included.     * Formula: $\eta^2_{partial} = \frac{SS_{effect}}{SS_{effect} + SS_{residual}}$.

Assumptions of Factorial ANOVA

• Numerical DV: Required.
• Independence of Observations: Met by study design.
• Normality: The DV must be normally distributed at each combined level (cell) of the IVs.     * Tested using Shapiro-Wilk test (swilk) for each cell.
• Homogeneity of Variance: Equal variance must exist across all subgroups/cells.     * Tested using Levene's test or the Robust Variance command (robvar) on a single variable representing all combined cells.

Stata Implementation

• Generating Combined Variables: Since some commands (like tabstat or robvar) cannot handle two IVs, a combined variable is created:     * egen combined_var = group(IV1 IV2), label

• Summary Statistics:     * tabstat DV, by(combined_var) s(n mean sd skew kurt)     * tab IV2 IV1, sum(DV) (gives a clean marginal mean table).

• Fitting the Model:     * anova DV IV1 IV2 IV1#IV2     * OR: anova DV IV1##IV2 (the double hash automatically includes main effects and interaction).

• Graphing Sequence:     1. anova DV IV1##IV2 (must run ANOVA first).     2. margins IV1#IV2 (calculates cell means for plotting).     3. marginsplot (generates the line graph).

• Testing Effect Size:     * estat esize (provides partial eta square values).

Educational Example: Anxiety and Age Study

• Paper Title: The effect of death anxiety and age on health-promoting behaviors.
• Variables:     * IV 1: Death Anxiety (Anxiety group vs. Control group).     * IV 2: Age (Young adults vs. Old adults).     * DV: Health-promoting lifestyle profile scores.
• Findings:     * Main Effect of Age: Not significant.     * Main Effect of Condition: Significant (Anxiety group reported more health-promoting behavior than Control).     * Interaction of Age and Condition: Significant. This "qualifies" the main effects and requires unpacking.

Questions & Discussion

• Question: Can the position of Factor A and B be swapped in the interaction phrasing (e.g., "Factor A modulates B" vs. "Factor B modulates A")?     * Response: Yes. Statistically, they are identical. The phrasing should be chosen based on which best fits the researcher's specific research question. Typically, both ways are not computed as they yield the same result.
• Question: What does the value in the bottom right corner of a marginal mean table represent?     * Response: That is the Grand Mean. It is the average of the entire participant group across all cells. While used in sum-of-squares hand calculations, it is generally less meaningful for direct result interpretation than cell and marginal means.
• Question: Why use Partial Eta Square instead of Eta Square?     * Response: If we only used Eta Square, adding more factors would naturally make each Eta Square value smaller because the denominator ($SS_{Total}$) stays the same while variance is partitioned more ways. Partial Eta Square allows for a more balanced comparison across studies by only including the specific effect and the error in the calculation.