Factorial Between-Participants ANOVA III & Higher-Order Designs

Flashcards cover manipulation checks, factorial terminology (main effects, interactions, simple effects), three-way ANOVA procedures (omnibus tests, follow-ups), effect sizes, structural models, degrees of freedom, interpretation cautions, example findings from the lecture’s hiring-bias study, and practical assessment reminders.

What is a manipulation check in experimental research?

A measure used to verify that participants interpreted or experienced the IV manipulation as intended.

When is a manipulation check especially important?

When the independent variable is psychological/unobservable and cannot be directly verified (e.g., distraction via loud music).

In a 3×2 alcohol (0,2,4 pints) × distraction (distracted, control) design, which omnibus test should be significant to confirm the distraction manipulation worked?

The main effect of distraction on the manipulation-check DV (scores should be higher in the distracted condition).

Which omnibus tests should be non-significant when validating the distraction manipulation in the same study?

The main effect of alcohol consumption and the Alcohol × Distraction interaction on the manipulation-check DV.

How are main effects defined in factorial ANOVA?

Differences between marginal means of one factor, averaging (collapsing) over all other factors.

What is a simple effect in two-way ANOVA?

The effect of one factor at each level of a second factor (comparison of cell means, ignoring other factors).

Define ‘simple comparison.’

A follow-up (usually t-test/contrast) that pinpoints which specific cell means differ within a significant simple effect.

What additional sources of variance are introduced in a three-way ANOVA compared with a two-way ANOVA?

One new main effect (Factor C), two new two-way interactions (A×C, B×C), and one three-way interaction (A×B×C).

How many omnibus tests exist in a 3-way ANOVA?

Seven: 3 main effects, 3 two-way interactions, 1 three-way interaction.

Write the structural model for a three-way between-participants ANOVA.

Xijkl = μ + αj + βk + γl + αβjk + αγjl + βγkl + αβγjkl + εijkl

What is the df formula for a main effect of a factor with ‘a’ levels?

df = a – 1

Give the df for an A×B×C interaction.

(a – 1)(b – 1)(c – 1)

How do you calculate error df in a fully crossed between-participants factorial design?

df_error = N – (number of cells)

Why can main effects be misleading when qualified by interactions?

Because the effect of a factor may differ across levels of another factor, making the averaged main effect unrepresentative.

In three-way ANOVA, are two-way interactions evaluated with cell means or special marginal means?

With marginal means averaged over the third factor (not raw cell means).

What does a significant three-way interaction indicate conceptually?

That a two-way interaction between two factors changes across levels of a third factor.

List the three steps for following up a significant three-way interaction.

1) Test simple interactions (two-way interactions at each level of the third factor). 2) If significant, test simple-simple effects (one factor at each level of the second factor within the third). 3) If factor >2 levels, conduct simple-simple comparisons to locate specific differences.

How do you decide which simple interactions to explore after a three-way interaction?

Base the choice on theoretical hypotheses and research questions—examine interactions most relevant to them.

In the hiring-bias example, what were the three factors?

Candidate Attractiveness (low, average, high), Candidate Gender (female, male), Participant Gender (female, male).

How many cells are in a 3 (Attractiveness) × 2 (Cand Gender) × 2 (Part Gender) design?

3 × 2 × 2 = 12 cells.

What simple interaction set was chosen for the hiring-bias example, and why?

Candidate Attractiveness × Candidate Gender at each level of Participant Gender, because the core question concerned how attractiveness effects depend on both candidate and evaluator gender.

Differentiate omnibus vs. simple two-way interactions in a three-way design.

Omnibus: interaction of two factors averaged over the third factor; Simple: interaction of two factors tested separately at each level of the third factor.

What is η² (eta squared)?

An effect-size measure for ANOVA representing proportion of total variance accounted for by a factor.

What is ω² (omega squared) and why might it be preferred?

An adjusted effect-size measure for ANOVA providing a less biased estimate of population effect size.

Which effect-size measure is commonly reported for t-tests?

Cohen’s d.

What is the guideline for plotting a three-way interaction line graph?

Y-axis: DV; X-axis: factor with most levels or theoretical priority; lines: second-most important factor; separate panels: least important factor.

Describe familywise error rate concern in higher-order follow-up testing.

Running many follow-up tests increases the probability of Type I errors across the family of comparisons.

Name two strategies to manage excessive follow-up tests in complex ANOVA.

(1) Conduct only tests that address hypotheses; (2) control error rate with adjustments (e.g., Bonferroni).

What practical skills will be assessed in Practical Test 3 (Week 6)?

Interpreting SPSS output for main-effect comparisons in a two-way ANOVA to decide significance and describe directions.

When following up a significant main effect with >2 levels, what test is run?

Main-effect comparisons (e.g., pairwise t-tests or linear contrasts) to see which marginal means differ.

If Factor = 2 levels, is a main-effect follow-up test needed?

No; direction is obvious from the two marginal means.

What is a ‘simple simple comparison’?

A pairwise comparison among cell means used after a significant simple simple effect (factor >2 levels) within a three-way design.

How many potential follow-up tests could exist in a full, exhaustive analysis of a 3-way ANOVA (per lecture example)?

57 tests (7 omnibus + 1 main-effect comparison + 14 simple effects + 7 simple interactions + 16 simple simple effects + 12 simple simple comparisons).

Why might researchers skip follow-ups for certain significant lower-order effects?

Because those effects are peripheral or qualified by higher-order interactions that render them less interpretable.

Explain the difference between η² and partial η² (ηp²).

η² uses total variance in the denominator; partial η² uses variance remaining after excluding other effects, making values larger.

Define ‘complex ANOVA.’

Another term for higher-order factorial designs (e.g., 3-way or 4-way ANOVA).

What is the purpose of a ‘common mistakes’ document for assessments?

To highlight frequent student errors (e.g., confusing main vs. simple effects) and guide correct reporting in practical tests.

Give an example of correctly reporting a significant simple effect.

‘Creativity scores were significantly higher in the distracted condition than the control condition for participants who consumed 0 pints, F(1, ??) = …, p = …’ (specifying both conditions).

What is the preferred order when interpreting effects in a complex ANOVA?

Start with the highest-order significant interaction, then move to lower-order interactions, then main effects.

What are ‘simple simple effects’ used for?

To understand how a factor’s effect varies across every combination of two other factors after a significant simple interaction.

Why must manipulation-check analyses use the same ANOVA design as the main DV?

To verify that the manipulation behaved consistently across all factors in the experimental structure.

If a three-level factor shows a significant main effect, but pairwise comparisons reveal only low ≠ average and low ≠ high, how would you summarise?

Participants differed only between low vs. average and low vs. high; average and high did not differ significantly.

In SPSS, which table provides omnibus F-tests for factorial ANOVA?

The ‘Tests of Between-Subjects Effects’ table.

Which SPSS procedure is used to get main-effect pairwise comparisons?

EMMEANS (Estimated Marginal Means) with pairwise or ‘Compare main effects’ options.

How is MS (mean square) calculated?

MS = SS / df.

Formula for F-statistic in ANOVA.

F = MSeffect / MSerror.

Why average over other factors when computing a main effect?

To isolate the pure effect of one factor independent of interactions.

What does ‘qualified by an interaction’ mean?

A main effect or lower-order interaction changes in magnitude or direction depending on another factor.

What is Deep vs. Normal breathing in the video-game example?

Levels of Factor 2: Breathing Method.

How many marginal means are compared for a 3-level main effect?

Three marginal means (one per level).

Give a caution when interpreting non-significant manipulation checks.

They suggest the manipulation may not have worked, threatening internal validity.

How many simple effects exist for Factor A in a 3-way design?

At each level of B and C (b+c) if B or C have 2+ levels; specifically, 2 at each level of the other factors. In the example: 2 (gender) + 2 (participant gender) = 4 simple effects for A.

What is the recommended graph layout for presenting simple interactions?

Separate graphs for the third factor, lines for the second factor, x-axis for the first factor.

Describe the hiring-bias result pattern for female participants rating female candidates.

Low = Average > High attractiveness ratings (high attractiveness penalised).

Describe the pattern for female participants rating male candidates.

Low < Average < High (higher attractiveness rewarded).

Describe the pattern for male participants rating male candidates.

Low = Average > High (high attractiveness penalised).

Describe the pattern for male participants rating female candidates.

Low < Average < High (higher attractiveness rewarded).

Which weeks in the course cover Regression topics following ANOVA?

Week 6 (Correlation + Standard Multiple Regression) and Week 7 (Standard & Hierarchical Multiple Regression).

What is a quasi-experiment?

A design where participants are not randomly assigned to all factor levels, often using pre-existing groups (e.g., participant gender).