Study Notes for Module 11 on Threats to Internal Validity

Module 11 Overview

  • Continuing discussion of experiments

  • Focus on threats to internal validity

  • Contents also connect to Module 12

Recording Context

  • Instructor recording from home for less distraction

  • Experiences issues with noise in on-campus recording

  • Casual remarks about personal life, including pet named Doctor Whiskers, emphasizing a relaxed learning atmosphere

Importance of Previous Learning

  • Review of material from Module 10 is recommended

  • Core concepts that are built upon include:

    • Identification of independent and dependent variables

    • The significance of random assignment

    • Difference between independent groups and within groups designs

    • Understanding of causal criteria: covariance, temporal precedence, internal validity

Structure of Module 11

  • Seven lectures planned for Module 11

  • Lecture 1: Overview of threats to internal validity

  • Continuing review of previously covered threats

    • New threats will also be introduced

    • All covered threats compiled in a comprehensive list:

    • Design confounds

    • Selection effects

    • Order effects

    • Maturation threats (New)

    • History threats (New)

    • Regression to the mean (New)

    • Attrition (New)

    • Testing effects (Previously covered)

    • Instrumentation threats (New)

    • Observer bias (Previously covered)

    • Demand characteristics (Previously covered)

    • Placebo effect (Potentially known, not directly covered)

Criteria for Causation

  • Familiarity with the three core criteria for establishing causation is crucial:

    1. Covariance

    • Proposed cause must change alongside the effect

    • Indicates correlation between variables

    • Correlational designs can demonstrate covariance

    1. Temporal Precedence

    • Establish that the cause occurs before the effect

    1. Internal Validity

    • Ensures that no third variable causes changes in the proposed cause and effect

  • Well-designed experiments help meet these criteria, thus supporting causal claims

  • Students are encouraged to trust experimental methods as means for causal inference due to assumed intent of researchers to establish such claims

Understanding Internal Validity

  • Definition: A study's adequacy in directing causal inferences based on findings

  • High internal validity generally found in experiments

  • Threats to internal validity can result from inadequacies in study design if systematic variations between independent and third variable occur

Detailed Threats to Internal Validity

1. Design Confounds

  • Occurs when a third variable systematically varies alongside the independent variable.

  • Example given:

    • Experiment with two testing environments (red room vs. blue room)

    • Uncertainty about results stemming from either room color or the independent variable (e.g., drug)

  • Importance of controlling all extraneous variables to isolate the effect of the independent variable

2. Selection Effects

  • Particularly relevant in independent groups designs (between subjects)

  • Risk arises when participants can choose their condition, leading to confounding based on personal preferences

  • Example of choice impacts on results, preventing definitive conclusions about independent variable effects

3. Order Effects

  • Pertains to within-subject designs, where participants are tested across all levels of the independent variable

  • Specific types of order effects include:

    • Practice Effects: Improvement over time due to repeated exposure to the test

    • Fatigue Effects: Deterioration in performance due to tiredness or boredom

    • Carryover Effects: Previous conditions affecting performance in subsequent conditions (e.g., tasting orange juice after toothpaste is negatively affected)

General Considerations on Threats to Validity

  • Acknowledgment of the complexities in scientific studies and prevalence of threats

  • Emphasis on the effectiveness of control groups to mitigate several identified threats

  • Discussion of the one-group pretest posttest design as particularly vulnerable to multiple threats

  • Explanation of why this design is often avoided in published studies and how understanding its flaws aids in assessing research quality

Conclusion and Forward Look

  • Upcoming lectures will introduce new threats: maturation threats and history threats

  • Reminder for students to review previous content for better understanding as topics progress