Multivariate Analysis of Variance and Factorial ANOVA
Fundamentals of Multivariate Analysis of Variance (MANOVA)
Definition:
- Multivariate Analysis of Variance (MANOVA) is an omnibus statistical test that evaluates group differences across two or more continuous dependent variables simultaneously.
Distinction from Univariate Techniques:
- Univariate analyses (such as -tests, one-way ANOVA, and factorial ANOVA) assess only a single dependent variable () at a time.
- MANOVA assesses multiple dependent variables () within the same statistical model.
Control of Family-Wise Type I Error Rate:
- Conducting multiple separate univariate ANOVAs inflates the cumulative probability of committing a Type I error (). For example, running three distinct ANOVAs each at accumulates independent error probabilities (), substantially increasing the overall likelihood of false positive findings.
- MANOVA preserves the family-wise Type I error rate at a constant across all dependent variables for the omnibus multivariate effect.
Taxonomy of MANOVA Designs:
- One-Way MANOVA: Incorporates exactly one categorical independent variable () having two or more levels, alongside multiple continuous dependent variables ().
- Factorial MANOVA: Incorporates two or more categorical independent variables () alongside multiple continuous dependent variables ().
- Crucial definitional note: A MANOVA is defined strictly by the presence of multiple dependent variables, not by having only one independent variable. Describing a MANOVA as possessing only one independent variable applies exclusively to the one-way MANOVA configuration.
Criteria for Dependent Variable Selection:
- Dependent variables integrated into a MANOVA must be conceptually related to one another (e.g., depression inventory scores and anxiety inventory scores in a pharmaceutical study contrasting a drug, placebo, and alternative active treatment).
- Conceptually unrelated variables (e.g., shoe size evaluated alongside psychiatric symptom measures) must never be combined into a single MANOVA.
- If a study records both related variables and an unrelated variable, the related variables should be analyzed together using a MANOVA, whereas the unrelated variable must be analyzed separately via an independent univariate ANOVA.
Experimental Study Design: Study Strategies and Learning Outcomes
Research Problem:
- An educational psychology investigation evaluated whether different study strategies systematically alter learning outcomes among undergraduate students.
- Standardized exposure: All participants were seated simultaneously in a single large lecture hall and presented with identical instructional lecture material.
Independent Variable Configuration:
- Independent Variable: Study Strategy (), organized into between-subjects levels:
- Thinking Condition (Level ): Participants were provided with review questions regarding the lecture material and directed exclusively to mentally formulate responses without writing anything down.
- Writing Condition (Level ): Participants were provided with identical review questions and instructed to write out complete textual answers.
- Talk Condition (Level ): Participants received the same prompts and were instructed to develop an instructional lecture to teach the material to their peers (eliciting a higher depth of cognitive processing), though they did not physically deliver the presentation.
- Independent Variable: Study Strategy (), organized into between-subjects levels:
Dependent Variables:
- Learning performance was operationalized through two distinct, conceptually related dependent measures:
- Recall Test (DV ): A strict retrieval test evaluating factual memory retention from the lecture.
- Application Test (DV ): An assessment measuring participants' capacity to generalize and apply lecture concepts to novel analytical problems.
- Learning performance was operationalized through two distinct, conceptually related dependent measures:
Dataset Specifications ( Cases):
- Independent Variable (
Study Strategy):- Cases through : Thinking Condition (coded as ).
- Cases through : Writing Condition (coded as ).
- Cases through : Talk Condition (coded as ).
- Recall Test Scores (DV ):
- Thinking (Cases to ):
- Writing (Cases to ):
- Talk (Cases to ):
- Application Test Scores (DV ):
- Thinking (Cases to ):
- Writing (Cases to ):
- Talk (Cases to ):
- Independent Variable (
SPSS Execution and Analytical Assumptions for MANOVA
Procedural Execution in SPSS:
- Navigate to
AnalyzeGeneral Linear ModelMultivariate. - Transfer
RecallandApplicationinto theDependent Variablesselection box. - Transfer
Study Strategyinto theFixed Factor(s)selection box. - Select
Post Hoc: MoveStudy Strategyto the post hoc tests box, and enable bothTukeyandGames-Howell. - Select
Options: CheckDescriptive statistics,Estimates of effect size(partial eta squared),Observed power, andHomogeneity tests. - Click
ContinueandOKto generate the multivariate output.
- Navigate to
Assessment of Statistical Assumptions:
- Box's M Test of Equality of Covariance Matrices:
- Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.
- Interpreted analogously to Levene's test of homogeneity of variance and Mauchly's test of sphericity: a non-significant result () indicates that the assumption is satisfied, whereas a significant result () indicates an assumption violation.
- Observed statistic: Box's , .
- Interpretation: Because , the assumption of equality of covariance matrices is met.
- Levene's Test of Equality of Error Variances:
- Assesses the assumption of homogeneity of variance across groups for each dependent variable individually.
- Both the recall test and the application test yielded non-significant Levene's tests, verifying that the assumption of homogeneity of variance was satisfied for both dependent variables.
- Box's M Test of Equality of Covariance Matrices:
Selection Rule for Multivariate Test Statistics:
- Wilks' Lambda (): Interpreted when the assumption of equality of covariance matrices (Box's M) is met.
- Pillai's Trace: Interpreted as an alternative, robust metric when the assumption of equality of covariance matrices is violated.
- Because Box's M test was non-significant, Wilks' Lambda is the appropriate statistic to report.
MANOVA Output Interpretation and Univariate Follow-Up Tests
Multivariate Effect Analysis:
- Statistical parameters: Wilks' , , , Partial .
- Substantive finding: There is a statistically significant multivariate effect of study strategy across the combined dependent measures of learning.
- Analytical stopping rule: Had the multivariate effect been non-significant (), the analysis would conclude immediately without examining univariate outputs. Because the multivariate effect is significant, analysis proceeds to univariate between-subjects effects.
Tests of Between-Subjects Effects (Univariate ANOVAs):
- Recall Test Performance:
- Statistical parameters: Between-groups , error , , .
- Substantive finding: There is a statistically significant effect of study strategy on recall performance.
- Application Test Performance:
- Statistical parameters: Between-groups , error , , .
- Substantive finding: There is a statistically significant effect of study strategy on application test scores.
- Recall Test Performance:
Pairwise Post Hoc Comparisons (Tukey's HSD):
- Because homogeneity of error variance was met, Tukey's HSD post hoc comparisons are interpreted (Games-Howell is reserved for violations of homogeneity).
- Post hoc tests are only inspected for dependent variables demonstrating a significant between-subjects effect.
- Recall Comparisons:
- Writing vs. Thinking: Statistically significant (); participants in the Writing condition exhibited significantly higher recall scores than those in the Thinking condition.
- Writing vs. Talk: Statistically significant (); participants in the Writing condition exhibited significantly higher recall scores than those in the Talk condition.
- Thinking vs. Talk: Non-significant difference.
- Application Comparisons:
- Writing vs. Thinking: Statistically significant (); participants in the Writing condition performed significantly better on application problems than those in the Thinking condition.
- Other pairwise comparisons for application: Non-significant.
Structural Composition of Results Reporting:
- Identify design: One-way between-subjects MANOVA with study strategy as the independent variable and recall and application as dependent variables.
- Report assumption tests: State Box's statistic (, ) to confirm equality of covariance matrices, and report Levene's tests confirming homogeneity of variance across both dependent variables.
- Report omnibus multivariate test: Wilks' , , degrees of freedom, -value, partial , and observed power.
- Report individual univariate ANOVAs: Separate -statistics, degrees of freedom, -values, and effect sizes for recall and application.
- Report post hoc pairwise tests: Describe group differences (directionality of means) for each significant univariate effect.
Two-Way Between-Subjects Factorial ANOVA ( Design)
Design Structure and Theoretical Principles:
- A between-subjects factorial ANOVA incorporates two independent variables, each containing two levels, evaluated against a single continuous dependent variable.
- Factor A (): Levels and .
- Factor B (): Levels and .
- Dependent Variable: A single continuous measure designated as
DV. - Cell count: Yields four () unique experimental conditions.
- Post hoc testing constraint: Because each factor possesses only two levels, post hoc pairwise comparisons (e.g., Tukey's HSD) are not conducted for main effects. A significant main effect of a two-level factor is interpreted directly by inspecting the marginal group means.
Dataset Entry ( Cases):
- Independent Variable
ACoding:- Cases through : Coded as Level ().
- Cases through : Coded as Level ().
- Independent Variable
BCoding:- Cases through : Coded as Level ().
- Cases through : Coded as Level ().
- Cases through : Coded as Level ().
- Cases through : Coded as Level ().
- Dependent Variable Scores (
DV):- Cases to ():
- Cases to ():
- Cases to ():
- Cases to ():
- Independent Variable
SPSS Execution Steps:
- Navigate to
AnalyzeGeneral Linear ModelUnivariate. - Move
DVinto theDependent Variablebox. - Move
AandBinto theFixed Factor(s)box. - In
Options, selectDescriptive statistics,Estimates of effect size,Observed power, andHomogeneity tests. - Execute analysis.
- Navigate to
Statistical Output and Effects Decomposition:
- Main Effect of Factor A: Non-significant ().
- Main Effect of Factor B: Non-significant.
- Interaction Effect (): Statistically significant ().
- Analytical Protocol for Significant Interactions:
- When an interaction effect is significant, simple effects analyses must be performed to break down the nature of the interaction (evaluating the simple main effect of Factor A at each individual level of Factor B, or the simple main effect of Factor B at each individual level of Factor A).
- In SPSS, conducting simple effects involves filtering data via
Select CasesorSplit Fileand executing independent-samples -tests on the groups (e.g., testing the effect of A for cases only, then testing the effect of A for cases only). - Crucial contingency: If the interaction term is non-significant, simple effects must neither be conducted nor referenced in the results narrative.
Typographical and Formatting Standards in Results Sections:
- Compound modifiers must be properly hyphenated: "two-way" and "between-subjects".
- Mathematical operators must be formatted with symmetrical leading and trailing spaces: e.g., and .
Short-Answer Conceptual Examination Review
Question 1: Definition of a Factorial ANOVA:
- A factorial ANOVA is an inferential statistical design comprising two or more categorical independent variables (factors) and a single continuous dependent variable.
- It tests for separate main effects for each independent variable as well as one or more interaction effects evaluating the combined influence of the independent variables.
Question 2: Information Inferred from Design Nomenclature:
- Two-Way Between-Subjects Factorial ANOVA:
- Two independent variables.
- Both factors are between-subjects variables (each participant experiences only one level of each factor).
- One continuous dependent variable.
- Evaluates two main effects and one two-way interaction.
- Repeated Measures Factorial ANOVA:
- Two independent variables, each containing two levels.
- Both factors are within-subjects/repeated measures variables (each participant is measured across all conditions).
- Yields four unique conditions.
- Because each factor has only two levels, significant main effects are interpreted solely by comparing marginal means without post hoc tests.
- Factorial ANOVA:
- Two independent variables: one having two levels and one having three levels.
- Yields six () unique experimental conditions.
- If the main effect of the three-level factor is significant, post hoc tests (e.g., Tukey's HSD) are mandatory to determine pairwise differences.
- Design naming vs. condition count: A design designated purely by number of factors (e.g., "two-way") indicates the number of IVs (), but does not specify the number of unique conditions. A design designated by factor levels (e.g., "") defines both the number of factors () and the exact number of unique conditions ().
- Two-Way Between-Subjects Factorial ANOVA:
Question 3: Definitions of Main Effects and Interactions:
- Main Effect: The isolated, overarching effect of one independent variable on the dependent variable, collapsing (averaging) across and ignoring all levels of all other independent variables.
- Interaction Effect: Occurs when the effect of one independent variable on the dependent variable differs significantly across the different levels of another independent variable (reflecting a conditional or combined effect).
Question 4: Analytical Decision Rules for Following Up Significant Effects:
- Given an experimental design (e.g., a factorial model testing Factor A with levels and Factor B with levels):
- Significant Main Effect of Factor A ( levels): Inspect the two marginal group means directly to determine directionality; no post hoc tests are performed.
- Significant Main Effect of Factor B ( levels): Conduct post hoc pairwise comparisons (e.g., Tukey's HSD) to isolate which specific pairs of group means differ significantly.
- Significant Interaction (): Conduct simple main effects analyses to test the effect of one factor at each specific level of the second factor (e.g., evaluating Factor A at Level , Factor A at Level , and Factor A at Level , or evaluating Factor B at Level and Factor B at Level ).
- Given an experimental design (e.g., a factorial model testing Factor A with levels and Factor B with levels):
Question 5: Definition and Purpose of MANOVA:
- Multivariate Analysis of Variance.
- An omnibus inferential test used to assess differences between experimental groups across multiple continuous dependent variables simultaneously.
- Primary mathematical benefit: Constrains the family-wise Type I error rate to a constant across all dependent variables, avoiding the error inflation associated with executing separate univariate ANOVAs.
- Structural scope: Can be configured as a one-way MANOVA ( IV, multiple DVs) or a factorial MANOVA (multiple IVs, multiple DVs).
Questions & Discussion
Factorial MANOVA Scope:
- Prompt: Clarification on whether factorial designs can incorporate both multiple independent variables and multiple dependent variables simultaneously.
- Response: Yes, a factorial MANOVA incorporates more than one independent variable and more than one dependent variable.
Independent Variable Levels for Study Strategy:
- Prompt: Clarification on the specific levels comprising the study strategy independent variable.
- Response: The three levels are Thinking, Writing, and Talk.
Dependent Variable Placement in SPSS:
- Prompt: Inquiry regarding whether Recall and Application are placed into the Dependent Variables box.
- Response: Yes, both Recall and Application are placed together into the Dependent Variables box.
Reporting Protocol for Significant Interactions:
- Prompt: Request to repeat the necessary analytical statement when an interaction effect is statistically significant.
- Response: When an interaction is significant, the report must state that simple effects analyses should be conducted to interpret the interaction, examining the effect of Factor A at each level of Factor B or the effect of Factor B at each level of Factor A. If the interaction is non-significant, simple effects are omitted entirely.
Exam Format for Follow-Up Procedures:
- Prompt: Request to repeat the exact structure of Question 4 on the examination.
- Response: An experimental design will be presented, and responses must delineate what effects are tested and explicitly state how each significant effect would be followed up (e.g., noting whether to inspect marginal means, conduct post hoc tests, or conduct simple main effects).