Factorial Designs, Ethics & QRPs

Factorial Design

An experiment with TWO OR MORE independent variables (also called factors). Allows researchers to test how multiple variables work together and whether they interact. Minimum requirements: at least 2 IVs, each with at least 2 levels. Example: Testing both caffeine (yes/no) and time of day (morning/evening) on reaction time.

Factor

Another word for independent variable (IV) in a factorial design. Example: In a study testing caffeine and time of day, 'caffeine' is one factor and 'time of day' is another factor.

Level

The different conditions or values of each independent variable (factor). Example: The factor 'caffeine' has 2 levels (caffeine vs. no caffeine). The factor 'time of day' has 2 levels (morning vs. evening).

2x2 Design (read as 'two by two')

A factorial design with 2 independent variables, each with 2 levels. Creates 4 total conditions (2 × 2 = 4). Example: Caffeine (yes/no) × Time of Day (morning/evening) = 4 conditions: (1) caffeine-morning, (2) caffeine-evening, (3) no caffeine-morning, (4) no caffeine-evening.

How to Calculate Total Conditions in Factorial Design

Multiply the number of levels of each IV together. Formula: Total conditions = (levels of IV1) × (levels of IV2) × (levels of IV3)... Examples: 2×2 = 4 conditions, 2×3 = 6 conditions, 2×2×2 = 8 conditions.

Main Effect

The overall effect of ONE independent variable on the dependent variable, averaging across all levels of the other IV(s). In simpler terms: Does this IV affect the DV on its own, ignoring the other IV? Check by comparing the MARGINAL MEANS for each level of the IV.

Marginal Means

The average for each level of one IV, collapsing across (averaging over) the other IV(s). Used to determine if there's a main effect. Example: In a 2×2 design, the marginal mean for 'caffeine' is the average of caffeine-morning and caffeine-evening combined.

Interaction

When the effect of one IV DEPENDS ON the level of the other IV. The 'it depends' rule: If you can say 'it depends' when describing the effect of one IV, you have an interaction. Visual cue: Non-parallel lines on a graph (lines that cross or diverge) indicate an interaction.

The 'It Depends' Rule for Interactions

If the answer to 'Does IV1 affect the DV?' is 'It depends on IV2,' then you have an interaction. Example: 'Does caffeine improve reaction time?' 'It depends on whether you're sleep-deprivedâ€"caffeine helps a lot when sleep-deprived but not much when well-rested.' = Interaction.

How to Spot an Interaction Visually

Look at a line graph of the results. PARALLEL LINES = NO interaction (the effect of one IV is the same at all levels of the other IV). NON-PARALLEL LINES (crossing or diverging) = YES, there's an interaction (the effect of one IV changes depending on the other IV).

Crossover Interaction

A special type of interaction where the lines on a graph literally CROSS, forming an X pattern. The effect of one IV is opposite at different levels of the other IV. Example: Introverts perform better in quiet, extroverts perform better with noiseâ€"the lines cross in the middle.

Participant Variable

A characteristic of participants that you MEASURE rather than MANIPULATE. Examples: age, gender, personality traits, clinical diagnosis. Important: When you use a participant variable, your design is QUASI-EXPERIMENTAL (not a true experiment), so you can't make strong causal claims.

Why Use Factorial Designs? (3 Main Reasons)

1) TEST THEORIES - Many theories predict that multiple factors work together. 2) FIND LIMITS OF AN EFFECT - Discover when an effect works and when it doesn't (boundary conditions). 3) INCREASE EXTERNAL VALIDITY - Multiple IVs create conditions more like the real world where multiple factors operate together.

How to Describe a Main Effect

Template: 'There is a main effect of [IV name], such that participants in the [level 1] condition [scored higher/lower/were faster/slower] (M = [mean]) than those in the [level 2] condition (M = [mean]).' Always specify WHICH level was higher/lower and cite the actual means.

How to Describe an Interaction

Template: 'There is an interaction between [IV1] and [IV2]. Specifically, [describe the pattern using actual numbers from each condition].' Example: 'The advantage of flashcards over re-reading was small in easy classes (5 points) but much larger in hard classes (15 points).'

Belmont Report

A 1979 report that established three core ethical principles for research with human participants: 1) RESPECT FOR PERSONS (autonomy and informed consent), 2) BENEFICENCE (minimize harm, maximize benefits), 3) JUSTICE (fair distribution of risks and benefits). Foundation for modern research ethics.

Respect for Persons

Ethical principle: Researchers must treat participants as autonomous agents capable of making their own decisions. Requirements: Obtain INFORMED CONSENT, allow participants to WITHDRAW at any time, protect vulnerable populations who can't give full consent (children, prisoners). No coercion.

Beneficence

Ethical principle: Researchers must protect participants from harm and ensure their well-being. Requirements: MINIMIZE RISKS, MAXIMIZE BENEFITS, ensure benefits outweigh risks. Researchers must actively protect participants' physical and psychological welfare throughout the study.

Justice

Ethical principle: The risks and benefits of research must be distributed FAIRLY across groups. Violation example: Tuskegee Syphilis Studyâ€"only African American men from low socioeconomic backgrounds bore the risks, but all groups would benefit from the findings. Researchers must not exploit vulnerable populations.

Informed Consent

The process of telling participants about the study's purpose, procedures, risks, and benefits BEFORE they agree to participate. Participants must understand what they're agreeing to and must participate VOLUNTARILY (no coercion). Part of Respect for Persons principle.

Deception in Research

When researchers intentionally mislead participants or withhold information about the study. Includes both ACTIVE LYING and LIES BY OMISSION. If deception is used, researchers MUST DEBRIEF participants afterwards, explaining the true purpose and methods. Still allowed but must be justified and minimized.

Debriefing

The process of explaining the study's true purpose and methods to participants AFTER the study is complete. Required when deception was used. Purpose: Ensure participants leave the study in the same state they entered, address any concerns, and educate them about the research.

Institutional Review Board (IRB)

A committee that reviews research proposals to ensure they meet ethical standards before the research can begin. Composition: Scientists AND community members (not just scientists). Requirement: Researchers must apply for IRB approval BEFORE conducting any study with human participants.

Tuskegee Syphilis Study

A notorious unethical study (1932-1972) where researchers studied untreated syphilis in African American men without their informed consent. Men were told they were receiving treatment but were actually left untreated, even after penicillin became available. Violated ALL THREE Belmont principles. Led to major ethics reforms.

HARKing (Hypothesizing After the Results are Known)

A questionable research practice where a researcher gets an UNEXPECTED finding but writes it up as if it was PREDICTED all along. Problem: Misrepresents the research process and inflates the appearance of theoretical support. Makes it seem like the theory predicted something it didn't.

P-Hacking

A questionable research practice where researchers analyze data in MULTIPLE DIFFERENT WAYS until they get a statistically significant result (p < .05), then only report the analysis that 'worked.' Problem: Inflates false positive rateâ€"you're more likely to find a 'significant' result just by chance when you try many analyses.

Underreporting Null Findings

A questionable research practice where researchers don't publish or report studies that found NO significant effect. Problem: Creates publication biasâ€"makes it seem like the evidence for a theory is STRONGER than it really is because all the 'failed' studies are hidden. Also called the 'file drawer problem.'

Preregistration

The practice of publicly documenting your research plan (hypotheses, methods, analysis plan) BEFORE collecting data. Purpose: Prevents HARKing and p-hacking by creating a permanent record of what you planned to do. Best done BEFORE DATA COLLECTION (not after).

Transparency in Research

The practice of openly sharing research materials, data, and methods so others can verify and build on your work. Includes: preregistration, sharing data and materials, reporting all measures and conditions (not just the ones that 'worked'). Goal: Make research more reproducible and trustworthy.

Questionable Research Practices (QRPs)

Research practices that are not outright fraud but undermine the integrity of science. Examples: HARKing, p-hacking, underreporting null findings, selectively reporting measures, stopping data collection when you get the result you want. Not illegal but ethically problematic.

What is NOT a Questionable Research Practice?

These are GOOD practices: 1) Reporting when you get a NON-significant result. 2) REPEATING a study to see if you get similar results (replication). 3) Using a LARGER sample size to increase power. 4) Analyzing data in multiple ways IF you report all analyses (not just the significant one).

Meta-Analysis

A statistical technique that combines results from MULTIPLE studies on the same topic to get an overall estimate of the effect size. Purpose: Get a more precise estimate than any single study can provide. Helps address the problem of underreporting null findings by systematically searching for all studies (published and unpublished).

The Three Possible Outcomes in a 2x2 Factorial Design

Outcome 1: TWO MAIN EFFECTS, NO INTERACTION (parallel linesâ€"both IVs affect DV independently). Outcome 2: TWO MAIN EFFECTS AND AN INTERACTION (non-parallel linesâ€"both IVs affect DV, and they combine in a complex way). Outcome 3: INTERACTION ONLY, NO MAIN EFFECTS (crossover interactionâ€"lines cross in the middle).

Common Mistake: Confusing Main Effects with Interactions

MAIN EFFECT = One IV affects the DV on average (ignoring the other IV). INTERACTION = The effect of one IV DEPENDS ON the other IV. If you can say 'it depends,' you're talking about an INTERACTION, not a main effect. Check marginal means for main effects; check if lines are parallel for interactions.