Multivariate Analysis of Variance and Factorial ANOVA

Fundamentals of Multivariate Analysis of Variance (MANOVA)

  • Definition:

    • Multivariate Analysis of Variance (MANOVA) is an omnibus statistical test that evaluates group differences across two or more continuous dependent variables simultaneously.
  • Distinction from Univariate Techniques:

    • Univariate analyses (such as tt-tests, one-way ANOVA, and factorial ANOVA) assess only a single dependent variable (DVDV) at a time.
    • MANOVA assesses multiple dependent variables (DVsDVs) within the same statistical model.
  • Control of Family-Wise Type I Error Rate:

    • Conducting multiple separate univariate ANOVAs inflates the cumulative probability of committing a Type I error (α\alpha). For example, running three distinct ANOVAs each at α=0.05\alpha = 0.05 accumulates independent error probabilities (0.05,0.05,0.050.05, 0.05, 0.05), substantially increasing the overall likelihood of false positive findings.
    • MANOVA preserves the family-wise Type I error rate at a constant α=0.05\alpha = 0.05 across all dependent variables for the omnibus multivariate effect.
  • Taxonomy of MANOVA Designs:

    • One-Way MANOVA: Incorporates exactly one categorical independent variable (IVIV) having two or more levels, alongside multiple continuous dependent variables (DVsDVs).
    • Factorial MANOVA: Incorporates two or more categorical independent variables (IVsIVs) alongside multiple continuous dependent variables (DVsDVs).
    • Crucial definitional note: A MANOVA is defined strictly by the presence of multiple dependent variables, not by having only one independent variable. Describing a MANOVA as possessing only one independent variable applies exclusively to the one-way MANOVA configuration.
  • Criteria for Dependent Variable Selection:

    • Dependent variables integrated into a MANOVA must be conceptually related to one another (e.g., depression inventory scores and anxiety inventory scores in a pharmaceutical study contrasting a drug, placebo, and alternative active treatment).
    • Conceptually unrelated variables (e.g., shoe size evaluated alongside psychiatric symptom measures) must never be combined into a single MANOVA.
    • If a study records both related variables and an unrelated variable, the related variables should be analyzed together using a MANOVA, whereas the unrelated variable must be analyzed separately via an independent univariate ANOVA.

Experimental Study Design: Study Strategies and Learning Outcomes

  • Research Problem:

    • An educational psychology investigation evaluated whether different study strategies systematically alter learning outcomes among undergraduate students.
    • Standardized exposure: All participants were seated simultaneously in a single large lecture hall and presented with identical instructional lecture material.
  • Independent Variable Configuration:

    • Independent Variable: Study Strategy (IVIV), organized into 33 between-subjects levels:
      • Thinking Condition (Level 11): Participants were provided with review questions regarding the lecture material and directed exclusively to mentally formulate responses without writing anything down.
      • Writing Condition (Level 22): Participants were provided with identical review questions and instructed to write out complete textual answers.
      • Talk Condition (Level 33): Participants received the same prompts and were instructed to develop an instructional lecture to teach the material to their peers (eliciting a higher depth of cognitive processing), though they did not physically deliver the presentation.
  • Dependent Variables:

    • Learning performance was operationalized through two distinct, conceptually related dependent measures:
      • Recall Test (DV 11): A strict retrieval test evaluating factual memory retention from the lecture.
      • Application Test (DV 22): An assessment measuring participants' capacity to generalize and apply lecture concepts to novel analytical problems.
  • Dataset Specifications (N=30N = 30 Cases):

    • Independent Variable (Study Strategy):
      • Cases 11 through 1010: Thinking Condition (coded as 11).
      • Cases 1111 through 2020: Writing Condition (coded as 22).
      • Cases 2121 through 3030: Talk Condition (coded as 33).
    • Recall Test Scores (DV 11):
      • Thinking (Cases 11 to 1010): 3,4,4,4,2,3,4,3,3,33, 4, 4, 4, 2, 3, 4, 3, 3, 3
      • Writing (Cases 1111 to 2020): 6,7,5,6,5,5,5,5,8,66, 7, 5, 6, 5, 5, 5, 5, 8, 6
      • Talk (Cases 2121 to 3030): 4,6,4,4,5,5,5,2,3,44, 6, 4, 4, 5, 5, 5, 2, 3, 4
    • Application Test Scores (DV 22):
      • Thinking (Cases 11 to 1010): 1,4,3,5,3,2,3,3,5,31, 4, 3, 5, 3, 2, 3, 3, 5, 3
      • Writing (Cases 1111 to 2020): 7,4,6,3,2,7,4,5,7,57, 4, 6, 3, 2, 7, 4, 5, 7, 5
      • Talk (Cases 2121 to 3030): 3,6,4,3,6,5,5,4,3,53, 6, 4, 3, 6, 5, 5, 4, 3, 5

SPSS Execution and Analytical Assumptions for MANOVA

  • Procedural Execution in SPSS:

    • Navigate to Analyze →\rightarrow General Linear Model →\rightarrow Multivariate.
    • Transfer Recall and Application into the Dependent Variables selection box.
    • Transfer Study Strategy into the Fixed Factor(s) selection box.
    • Select Post Hoc: Move Study Strategy to the post hoc tests box, and enable both Tukey and Games-Howell.
    • Select Options: Check Descriptive statistics, Estimates of effect size (partial eta squared), Observed power, and Homogeneity tests.
    • Click Continue and OK to generate the multivariate output.
  • Assessment of Statistical Assumptions:

    • Box's M Test of Equality of Covariance Matrices:
      • Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.
      • Interpreted analogously to Levene's test of homogeneity of variance and Mauchly's test of sphericity: a non-significant result (p>0.05p > 0.05) indicates that the assumption is satisfied, whereas a significant result (p≤0.05p \le 0.05) indicates an assumption violation.
      • Observed statistic: Box's M=6.980M = 6.980, p=0.398p = 0.398.
      • Interpretation: Because p=0.398>0.05p = 0.398 > 0.05, the assumption of equality of covariance matrices is met.
    • Levene's Test of Equality of Error Variances:
      • Assesses the assumption of homogeneity of variance across groups for each dependent variable individually.
      • Both the recall test and the application test yielded non-significant Levene's tests, verifying that the assumption of homogeneity of variance was satisfied for both dependent variables.
  • Selection Rule for Multivariate Test Statistics:

    • Wilks' Lambda (Λ\Lambda): Interpreted when the assumption of equality of covariance matrices (Box's M) is met.
    • Pillai's Trace: Interpreted as an alternative, robust metric when the assumption of equality of covariance matrices is violated.
    • Because Box's M test was non-significant, Wilks' Lambda is the appropriate statistic to report.

MANOVA Output Interpretation and Univariate Follow-Up Tests

  • Multivariate Effect Analysis:

    • Statistical parameters: Wilks' Λ=0.421\Lambda = 0.421, F(4,52)=7.028F(4, 52) = 7.028, p=0.001p = 0.001, Partial η2=0.351\eta^2 = 0.351.
    • Substantive finding: There is a statistically significant multivariate effect of study strategy across the combined dependent measures of learning.
    • Analytical stopping rule: Had the multivariate effect been non-significant (p>0.05p > 0.05), the analysis would conclude immediately without examining univariate outputs. Because the multivariate effect is significant, analysis proceeds to univariate between-subjects effects.
  • Tests of Between-Subjects Effects (Univariate ANOVAs):

    • Recall Test Performance:
      • Statistical parameters: Between-groups df=2df = 2, error df=27df = 27, F(2,27)=17.11F(2, 27) = 17.11, p<0.001p < 0.001.
      • Substantive finding: There is a statistically significant effect of study strategy on recall performance.
    • Application Test Performance:
      • Statistical parameters: Between-groups df=2df = 2, error df=27df = 27, F(2,27)=4.20F(2, 27) = 4.20, p=0.026p = 0.026.
      • Substantive finding: There is a statistically significant effect of study strategy on application test scores.
  • Pairwise Post Hoc Comparisons (Tukey's HSD):

    • Because homogeneity of error variance was met, Tukey's HSD post hoc comparisons are interpreted (Games-Howell is reserved for violations of homogeneity).
    • Post hoc tests are only inspected for dependent variables demonstrating a significant between-subjects effect.
    • Recall Comparisons:
      • Writing vs. Thinking: Statistically significant (p<0.05p < 0.05); participants in the Writing condition exhibited significantly higher recall scores than those in the Thinking condition.
      • Writing vs. Talk: Statistically significant (p<0.05p < 0.05); participants in the Writing condition exhibited significantly higher recall scores than those in the Talk condition.
      • Thinking vs. Talk: Non-significant difference.
    • Application Comparisons:
      • Writing vs. Thinking: Statistically significant (p<0.05p < 0.05); participants in the Writing condition performed significantly better on application problems than those in the Thinking condition.
      • Other pairwise comparisons for application: Non-significant.
  • Structural Composition of Results Reporting:

    • Identify design: One-way between-subjects MANOVA with study strategy as the independent variable and recall and application as dependent variables.
    • Report assumption tests: State Box's MM statistic (M=6.980M = 6.980, p=0.398p = 0.398) to confirm equality of covariance matrices, and report Levene's tests confirming homogeneity of variance across both dependent variables.
    • Report omnibus multivariate test: Wilks' Λ\Lambda, FF, degrees of freedom, pp-value, partial η2\eta^2, and observed power.
    • Report individual univariate ANOVAs: Separate FF-statistics, degrees of freedom, pp-values, and effect sizes for recall and application.
    • Report post hoc pairwise tests: Describe group differences (directionality of means) for each significant univariate effect.

Two-Way Between-Subjects Factorial ANOVA (2×22 \times 2 Design)

  • Design Structure and Theoretical Principles:

    • A 2×22 \times 2 between-subjects factorial ANOVA incorporates two independent variables, each containing two levels, evaluated against a single continuous dependent variable.
    • Factor A (IV1IV_1): Levels A1A_1 and A2A_2.
    • Factor B (IV2IV_2): Levels B1B_1 and B2B_2.
    • Dependent Variable: A single continuous measure designated as DV.
    • Cell count: Yields four (2×2=42 \times 2 = 4) unique experimental conditions.
    • Post hoc testing constraint: Because each factor possesses only two levels, post hoc pairwise comparisons (e.g., Tukey's HSD) are not conducted for main effects. A significant main effect of a two-level factor is interpreted directly by inspecting the marginal group means.
  • Dataset Entry (N=20N = 20 Cases):

    • Independent Variable A Coding:
      • Cases 11 through 1010: Coded as Level A1A_1 (11).
      • Cases 1111 through 2020: Coded as Level A2A_2 (22).
    • Independent Variable B Coding:
      • Cases 11 through 55: Coded as Level B1B_1 (11).
      • Cases 66 through 1010: Coded as Level B2B_2 (22).
      • Cases 1111 through 1515: Coded as Level B1B_1 (11).
      • Cases 1616 through 2020: Coded as Level B2B_2 (22).
    • Dependent Variable Scores (DV):
      • Cases 11 to 55 (A1,B1A_1, B_1): 8,9,7,9,78, 9, 7, 9, 7
      • Cases 66 to 1010 (A1,B2A_1, B_2): 5,4,6,6,45, 4, 6, 6, 4
      • Cases 1111 to 1515 (A2,B1A_2, B_1): 4,4,6,6,54, 4, 6, 6, 5
      • Cases 1616 to 2020 (A2,B2A_2, B_2): 7,7,9,9,87, 7, 9, 9, 8
  • SPSS Execution Steps:

    • Navigate to Analyze →\rightarrow General Linear Model →\rightarrow Univariate.
    • Move DV into the Dependent Variable box.
    • Move A and B into the Fixed Factor(s) box.
    • In Options, select Descriptive statistics, Estimates of effect size, Observed power, and Homogeneity tests.
    • Execute analysis.
  • Statistical Output and Effects Decomposition:

    • Main Effect of Factor A: Non-significant (p=1.000p = 1.000).
    • Main Effect of Factor B: Non-significant.
    • Interaction Effect (A×BA \times B): Statistically significant (p<0.001p < 0.001).
    • Analytical Protocol for Significant Interactions:
      • When an interaction effect is significant, simple effects analyses must be performed to break down the nature of the interaction (evaluating the simple main effect of Factor A at each individual level of Factor B, or the simple main effect of Factor B at each individual level of Factor A).
      • In SPSS, conducting simple effects involves filtering data via Select Cases or Split File and executing independent-samples tt-tests on the groups (e.g., testing the effect of A for B1B_1 cases only, then testing the effect of A for B2B_2 cases only).
      • Crucial contingency: If the interaction term is non-significant, simple effects must neither be conducted nor referenced in the results narrative.
  • Typographical and Formatting Standards in Results Sections:

    • Compound modifiers must be properly hyphenated: "two-way" and "between-subjects".
    • Mathematical operators must be formatted with symmetrical leading and trailing spaces: e.g., p<0.001p < 0.001 and F=7.028F = 7.028.

Short-Answer Conceptual Examination Review

  • Question 1: Definition of a Factorial ANOVA:

    • A factorial ANOVA is an inferential statistical design comprising two or more categorical independent variables (factors) and a single continuous dependent variable.
    • It tests for separate main effects for each independent variable as well as one or more interaction effects evaluating the combined influence of the independent variables.
  • Question 2: Information Inferred from Design Nomenclature:

    • Two-Way Between-Subjects Factorial ANOVA:
      • Two independent variables.
      • Both factors are between-subjects variables (each participant experiences only one level of each factor).
      • One continuous dependent variable.
      • Evaluates two main effects and one two-way interaction.
    • 2×22 \times 2 Repeated Measures Factorial ANOVA:
      • Two independent variables, each containing two levels.
      • Both factors are within-subjects/repeated measures variables (each participant is measured across all conditions).
      • Yields four unique conditions.
      • Because each factor has only two levels, significant main effects are interpreted solely by comparing marginal means without post hoc tests.
    • 2×32 \times 3 Factorial ANOVA:
      • Two independent variables: one having two levels and one having three levels.
      • Yields six (2×3=62 \times 3 = 6) unique experimental conditions.
      • If the main effect of the three-level factor is significant, post hoc tests (e.g., Tukey's HSD) are mandatory to determine pairwise differences.
    • Design naming vs. condition count: A design designated purely by number of factors (e.g., "two-way") indicates the number of IVs (22), but does not specify the number of unique conditions. A design designated by factor levels (e.g., "2×32 \times 3") defines both the number of factors (22) and the exact number of unique conditions (66).
  • Question 3: Definitions of Main Effects and Interactions:

    • Main Effect: The isolated, overarching effect of one independent variable on the dependent variable, collapsing (averaging) across and ignoring all levels of all other independent variables.
    • Interaction Effect: Occurs when the effect of one independent variable on the dependent variable differs significantly across the different levels of another independent variable (reflecting a conditional or combined effect).
  • Question 4: Analytical Decision Rules for Following Up Significant Effects:

    • Given an experimental design (e.g., a 2×32 \times 3 factorial model testing Factor A with 22 levels and Factor B with 33 levels):
      • Significant Main Effect of Factor A (22 levels): Inspect the two marginal group means directly to determine directionality; no post hoc tests are performed.
      • Significant Main Effect of Factor B (33 levels): Conduct post hoc pairwise comparisons (e.g., Tukey's HSD) to isolate which specific pairs of group means differ significantly.
      • Significant Interaction (A×BA \times B): Conduct simple main effects analyses to test the effect of one factor at each specific level of the second factor (e.g., evaluating Factor A at Level B1B_1, Factor A at Level B2B_2, and Factor A at Level B3B_3, or evaluating Factor B at Level A1A_1 and Factor B at Level A2A_2).
  • Question 5: Definition and Purpose of MANOVA:

    • Multivariate Analysis of Variance.
    • An omnibus inferential test used to assess differences between experimental groups across multiple continuous dependent variables simultaneously.
    • Primary mathematical benefit: Constrains the family-wise Type I error rate to a constant α=0.05\alpha = 0.05 across all dependent variables, avoiding the error inflation associated with executing separate univariate ANOVAs.
    • Structural scope: Can be configured as a one-way MANOVA (11 IV, multiple DVs) or a factorial MANOVA (multiple IVs, multiple DVs).

Questions & Discussion

  • Factorial MANOVA Scope:

    • Prompt: Clarification on whether factorial designs can incorporate both multiple independent variables and multiple dependent variables simultaneously.
    • Response: Yes, a factorial MANOVA incorporates more than one independent variable and more than one dependent variable.
  • Independent Variable Levels for Study Strategy:

    • Prompt: Clarification on the specific levels comprising the study strategy independent variable.
    • Response: The three levels are Thinking, Writing, and Talk.
  • Dependent Variable Placement in SPSS:

    • Prompt: Inquiry regarding whether Recall and Application are placed into the Dependent Variables box.
    • Response: Yes, both Recall and Application are placed together into the Dependent Variables box.
  • Reporting Protocol for Significant Interactions:

    • Prompt: Request to repeat the necessary analytical statement when an interaction effect is statistically significant.
    • Response: When an interaction is significant, the report must state that simple effects analyses should be conducted to interpret the interaction, examining the effect of Factor A at each level of Factor B or the effect of Factor B at each level of Factor A. If the interaction is non-significant, simple effects are omitted entirely.
  • Exam Format for Follow-Up Procedures:

    • Prompt: Request to repeat the exact structure of Question 4 on the examination.
    • Response: An experimental design will be presented, and responses must delineate what effects are tested and explicitly state how each significant effect would be followed up (e.g., noting whether to inspect marginal means, conduct post hoc tests, or conduct simple main effects).