chapter 10
Overview of Experimental Design
The transcript covers the learning objectives and concepts related to experiments in psychological science, and emphasizes understanding causal claims.
Learning Objectives
Apply Causation Criteria
Apply the three criteria for establishing causation to experiments and explain their support for causal claims.
Identify an experiment's independent, dependent, and control variables.
Classify experiments as independent-groups or within-groups designs and understand the rationale behind each.
Evaluate three potential threats to internal validity:
Design confounds
Selection effects
Order effects
Interrogate experimental designs utilizing the four validities.
Causal Claims
Definition
Causal Claim: The most assertive claim a scientist can make. Causal claims utilize stronger verbs such as "makes, influences, or affects," replacing phrases like "related to" or "associated with."
Example: Adults should model behaviors in front of infants to shape social norms.
Importance of Experiments
Experiments are essential for investigating causal relationships.
Interventions in psychological studies hinge on solid experimental research.
Examples of Simple Experiments
Example 1: Publicizing Norms
Objective: Foster inclusive climates through the publicizing of positive social norms.
Researchers: Sohad Murrar, Mitchell Campbell, Markus Brauer (2020) at the University of Wisconsin (UW) Madison.
Study Design
Phase 1: Preliminary survey to gauge existing positive social norms regarding diversity on campus.
Results:
93% of students agreed that diversity was embraced.
84% agreed to have their photo used in a poster promoting this message.
Phase 2: Conducting the experiment with 163 participants in a lab setting.
Environment: Waiting room displaying either the norms poster or a flu shot poster (manipulated variable).
Procedure: Each participant stayed in the waiting room for 5-7 minutes, then performed a filler memory task and completed an inclusive climate attitudes survey.
Results
Higher scores on the inclusive climate measure were observed for individuals who saw the norms poster compared to the flu shot poster.
Repetition: Murrar et al. conducted four additional studies showing similar results, thereby supporting a causal claim that the norms intervention promoted positive attitudes and perceptions.
Example 2: Motivating Babies
Objective: Explore how watching adult persistence influences infant behavior.
Researchers: Leonard et al. (2017) studied over 100 babies aged 13-18 months at a Boston Children’s Museum.
Study Design
Babies were assigned to either an effort condition (where models demonstrate persistence) or a no-effort condition (models show ease).
Outcomes were measured based on how many times babies pressed an inert toy button.
Results
Babies in the effort condition pressed the button approximately 11 times more than those in the no-effort condition, demonstrating the effect of adult modeling on infant behavior.
Key Concepts in Experimental Design
Experimental Variables
Definition of Experiment: Manipulation of at least one variable while measuring another.
Types of Variables:
Manipulated Variable: Controlled and altered by researchers (e.g., type of poster in Murrar’s study).
Measured Variable: Recorded behaviors or attitudes (e.g., responses in surveys).
Diagram of Variables
Independent Variable: Causal variable (e.g., model's effort behavior).
Dependent Variable: Measured variable (e.g., number of button presses).
Control Variables: Constants maintained across conditions to isolate effects (e.g., waiting room environment).
Covariance, Temporal Precedence, and Internal Validity
Experiments establish:
Covariance: Differences in outcome depending on manipulation (e.g., different attitudes based on poster).
Temporal Precedence: The causal variable precedes the outcome (advertisement before attitudes).
Internal Validity: Ruling out external variables as alternative explanations (methodological rigor).
Control of Internal Validity Threats
Design Confounds: Accidental influence of variables (e.g., conditions of the waiting room).
Selection Effects: Systematic differences between groups; use of random assignment to mitigate (e.g., coin flip).
Order Effects: Occurs in within-groups designs; counterbalancing reduces these effects.
Types of Experimental Designs
Independent-Groups Designs
Definition: Different participants are assigned to different conditions of the independent variable.
Types:
Posttest-Only Design: Assessing dependent variable post-manipulation.
Pretest/Posttest Design: Assessing changes pre- and post-manipulation to confirm group equivalency.
Within-Groups Designs
Definition: Same participants undergo all conditions of the independent variable.
Types:
Repeated-Measures Design: Participants rated preferences post each condition exposure.
Concurrent-Measures Design: Assessing behavioral preferences across conditions at the same time.
Advantages & Disadvantages of Designs
Within-Groups: Control over participant variable effects; potentially fewer participants needed.
Independent-Groups: Reduced carryover risks, but possibly more participants required.
Validities in Experimentation
Interrogation with Four Validities
Construct Validity: How accurately variables were manipulated and measured.
External Validity: Generalizability of findings across different populations and settings.
Statistical Validity: Significance and effect sizes of findings, including CIs and replication.
Internal Validity: Control of confounds and isolation of causal variables.
Statistical Validity and Replication
Detailed measures of effect size are essential (e.g., d, confidence intervals).
Replication across studies reinforces findings.
Summary
Causal claims must be supported by robust experimental designs, including clearly defined variables, controlled settings, and comprehensive analyses of internal and external validity.
Proper understanding of independent and dependent variables, designs, and threats to validity are critical for validating research outcomes.
Glossary Terms:
Social norm: Common beliefs in a group.
Experiment: Study manipulating at least one variable and measuring another.
Manipulated variable: Controlled part of the experiment.
Measured variable: The outcome recorded.
Independent variable: What is manipulated.
Dependent variable: The outcome measured.
Control variable: Factors kept constant to avoid impact on results.
Comparison group: A baseline or control condition for comparison.
Confound: A potential alternative explanation for observed effects.