chapter 10

Overview of Experimental Design

  • The transcript covers the learning objectives and concepts related to experiments in psychological science, and emphasizes understanding causal claims.

Learning Objectives

Apply Causation Criteria

  • Apply the three criteria for establishing causation to experiments and explain their support for causal claims.

  • Identify an experiment's independent, dependent, and control variables.

  • Classify experiments as independent-groups or within-groups designs and understand the rationale behind each.

  • Evaluate three potential threats to internal validity:

    • Design confounds

    • Selection effects

    • Order effects

  • Interrogate experimental designs utilizing the four validities.

Causal Claims

Definition

  • Causal Claim: The most assertive claim a scientist can make. Causal claims utilize stronger verbs such as "makes, influences, or affects," replacing phrases like "related to" or "associated with."

  • Example: Adults should model behaviors in front of infants to shape social norms.

Importance of Experiments

  • Experiments are essential for investigating causal relationships.

  • Interventions in psychological studies hinge on solid experimental research.

Examples of Simple Experiments

Example 1: Publicizing Norms

  • Objective: Foster inclusive climates through the publicizing of positive social norms.

  • Researchers: Sohad Murrar, Mitchell Campbell, Markus Brauer (2020) at the University of Wisconsin (UW) Madison.

Study Design
  • Phase 1: Preliminary survey to gauge existing positive social norms regarding diversity on campus.

    • Results:

    • 93% of students agreed that diversity was embraced.

    • 84% agreed to have their photo used in a poster promoting this message.

  • Phase 2: Conducting the experiment with 163 participants in a lab setting.

    • Environment: Waiting room displaying either the norms poster or a flu shot poster (manipulated variable).

    • Procedure: Each participant stayed in the waiting room for 5-7 minutes, then performed a filler memory task and completed an inclusive climate attitudes survey.

Results
  • Higher scores on the inclusive climate measure were observed for individuals who saw the norms poster compared to the flu shot poster.

  • Repetition: Murrar et al. conducted four additional studies showing similar results, thereby supporting a causal claim that the norms intervention promoted positive attitudes and perceptions.

Example 2: Motivating Babies

  • Objective: Explore how watching adult persistence influences infant behavior.

  • Researchers: Leonard et al. (2017) studied over 100 babies aged 13-18 months at a Boston Children’s Museum.

Study Design
  • Babies were assigned to either an effort condition (where models demonstrate persistence) or a no-effort condition (models show ease).

  • Outcomes were measured based on how many times babies pressed an inert toy button.

Results
  • Babies in the effort condition pressed the button approximately 11 times more than those in the no-effort condition, demonstrating the effect of adult modeling on infant behavior.

Key Concepts in Experimental Design

Experimental Variables

  • Definition of Experiment: Manipulation of at least one variable while measuring another.

  • Types of Variables:

    • Manipulated Variable: Controlled and altered by researchers (e.g., type of poster in Murrar’s study).

    • Measured Variable: Recorded behaviors or attitudes (e.g., responses in surveys).

Diagram of Variables
  • Independent Variable: Causal variable (e.g., model's effort behavior).

  • Dependent Variable: Measured variable (e.g., number of button presses).

  • Control Variables: Constants maintained across conditions to isolate effects (e.g., waiting room environment).

Covariance, Temporal Precedence, and Internal Validity

  • Experiments establish:

    • Covariance: Differences in outcome depending on manipulation (e.g., different attitudes based on poster).

    • Temporal Precedence: The causal variable precedes the outcome (advertisement before attitudes).

    • Internal Validity: Ruling out external variables as alternative explanations (methodological rigor).

Control of Internal Validity Threats

  • Design Confounds: Accidental influence of variables (e.g., conditions of the waiting room).

  • Selection Effects: Systematic differences between groups; use of random assignment to mitigate (e.g., coin flip).

  • Order Effects: Occurs in within-groups designs; counterbalancing reduces these effects.

Types of Experimental Designs

Independent-Groups Designs

  • Definition: Different participants are assigned to different conditions of the independent variable.

  • Types:

    • Posttest-Only Design: Assessing dependent variable post-manipulation.

    • Pretest/Posttest Design: Assessing changes pre- and post-manipulation to confirm group equivalency.

Within-Groups Designs

  • Definition: Same participants undergo all conditions of the independent variable.

  • Types:

    • Repeated-Measures Design: Participants rated preferences post each condition exposure.

    • Concurrent-Measures Design: Assessing behavioral preferences across conditions at the same time.

Advantages & Disadvantages of Designs

  • Within-Groups: Control over participant variable effects; potentially fewer participants needed.

  • Independent-Groups: Reduced carryover risks, but possibly more participants required.

Validities in Experimentation

Interrogation with Four Validities

  • Construct Validity: How accurately variables were manipulated and measured.

  • External Validity: Generalizability of findings across different populations and settings.

  • Statistical Validity: Significance and effect sizes of findings, including CIs and replication.

  • Internal Validity: Control of confounds and isolation of causal variables.

Statistical Validity and Replication

  • Detailed measures of effect size are essential (e.g., d, confidence intervals).

  • Replication across studies reinforces findings.

Summary

  • Causal claims must be supported by robust experimental designs, including clearly defined variables, controlled settings, and comprehensive analyses of internal and external validity.

  • Proper understanding of independent and dependent variables, designs, and threats to validity are critical for validating research outcomes.

Glossary Terms:

  • Social norm: Common beliefs in a group.

  • Experiment: Study manipulating at least one variable and measuring another.

  • Manipulated variable: Controlled part of the experiment.

  • Measured variable: The outcome recorded.

  • Independent variable: What is manipulated.

  • Dependent variable: The outcome measured.

  • Control variable: Factors kept constant to avoid impact on results.

  • Comparison group: A baseline or control condition for comparison.

  • Confound: A potential alternative explanation for observed effects.