Experiments

Nomothetic Causality in Experimental Research

• Core Advantage of Experiments: Establishing causal relationships between variables under controlled conditions.

• Three Criteria for Nomothetic Causality:

  • Correlation: Attributes of the independent variable ($x$) must systematically covary or be associated with attributes of the dependent variable ($y$).

  • Time Order: The independent variable ($x$) must temporarily precede the dependent variable ($y$) in chronological sequence.

  • Non-Spuriousness: The empirical relationship between $x$ and $y$ cannot be explained by or attributed to a third, extraneous confounding variable.

The Classical Experimental Design

• Core Elements of Classical Design:

  • Random Assignment: Participants are probabilistically assigned to experimental or control groups to create initial baseline equivalence.

  • Experimental Group (Treatment Group): The group subjected to the independent variable or intervention.

  • Control Group: The group held at baseline without receiving the independent variable or intervention.

  • Pre-Test: Baseline measurement of the dependent variable ($y$) across both experimental and control groups prior to intervention.

  • Stimulus / Treatment / Intervention: Administration of the independent variable ($x$) to the experimental group.

  • Post-Test: Re-measurement of the dependent variable ($y$) across both groups following the intervention to evaluate changes.

• Classical Design Matrix:

  • Group 1 (Experimental): Random Assignment | Pre-Test: Yes | Intervention: Yes | Post-Test: Yes

  • Group 2 (Control): Random Assignment | Pre-Test: Yes | Intervention: No | Post-Test: Yes

Equating Groups Through Matching Techniques

• Purpose of Matching: Used to equate experimental and control groups on critical confounding variables when random assignment alone needs structural reinforcement or is non-viable.

• Precision Matching (One-to-One Matching):

  • Subjects are paired individually based on specific, critical background characteristics (e.g., age, gender, education level).

  • One member of each pair is assigned to the experimental group, and the other is assigned to the control group.

• Frequency Distribution Matching (Matching by Averages):

  • Equates the overall sample distributions or statistical averages of key variables across the groups rather than individual pairing.

  • Ensures that central tendencies (e.g., mean age or baseline scores) are approximately equal across groups.

Internal Validity and Threats to Causal Inference

• Definition of Internal Validity: The degree of confidence that an experimental design isolates and measures the precise relationship between the independent variable ($x$) and dependent variable ($y$) without interference from extraneous factors.

• Threats to Internal Validity:

  • Selection Bias: Systematic baseline differences in subject selection or self-selection into groups.

  • History: Unplanned external events occurring during the course of the experiment that influence the dependent variable.

  • Maturation: Natural biological, psychological, or experiential changes occurring within participants over time.

  • Mortality (Experimental Attrition): Non-random participant dropouts during the study.

  • Regression to the Mean: The natural statistical tendency of extreme initial pre-test scores to move closer to the average on re-testing.

  • Diffusion of Treatments: Control group participants unintentionally or intentionally receiving elements of the intervention meant for the experimental group.

  • Instrumentation: Variations or unreliability in measurement tools, observers, or scoring procedures between pre-test and post-test.

  • Testing Effects: Pre-test sensitization, where taking an initial test alters or primes participant performance on subsequent post-tests.

External Validity and Generalizability

• Definition of External Validity: The extent to which experimental findings can be generalized across distinct settings, natural environments, and larger target populations.

• Factors Decreasing External Validity:

  • Low Generalizability to Natural Settings: Artificial or highly contrived laboratory environments that fail to mirror real-world conditions.

  • Reactivity: Behavioral changes induced by subject awareness of being studied or observed by researchers (e.g., Hawthorne effect).

  • Low Generalizability to Larger Populations: Lack of sample representativeness relative to the broader target population.

Strategies for Mitigating Validity Threats

• Replication: Executing studies across different samples, operationalizations, and environments strengthens both internal and external validity.

• Representative Sampling: Employing probability sampling techniques ensures higher population generalizability.

• Integrating Lab and Field Experiments: Combining laboratory studies with naturalistic field experiments bridges internal experimental control with real-world external generalizability.

• Blinding Protocols:

  • Single-Blind: Participants are unaware of their group assignment (experimental vs. control).

  • Double-Blind: Neither participants nor researchers interacting with them know group assignments, eliminating reactivity and researcher bias.

• Varying Design Structures: Utilizing specialized experimental designs (such as Solomon 4-Group) to isolate and control for pre-test effects.

True Experimental Variations

• Requirement for True Experiments: Must include Random Assignment of participants.

• Solomon 4-Group Design:

  • Group 1: Random Assignment | Pre-Test: Yes | Intervention: Yes | Post-Test: Yes

  • Group 2: Random Assignment | Pre-Test: Yes | Intervention: No | Post-Test: Yes

  • Group 3: Random Assignment | Pre-Test: No | Intervention: Yes | Post-Test: Yes

  • Group 4: Random Assignment | Pre-Test: No | Intervention: No | Post-Test: Yes

  • Analytical Advantage: Evaluates main intervention effects while disentangling pre-test testing effects and testing-treatment interactions.

• Two-Group Post-Test Only Design (Campbell-Stanley Design):

  • Group 1: Random Assignment | Pre-Test: No | Intervention: Yes | Post-Test: Yes

  • Group 2: Random Assignment | Pre-Test: No | Intervention: No | Post-Test: Yes

  • Analytical Advantage: Completely eliminates pre-test testing threats while maintaining internal validity through random assignment.

Field Experiments and Audit Studies: "The Mark of a Criminal Record"

• Naturalistic Experimental Features:

  • Conducted in real-world conditions rather than laboratory settings.

  • Manipulates variables in the natural world to evaluate authentic behavioral responses.

  • Frequently incorporates field deception or audit methodologies.

• Audit Study Analysis: "The Mark of a Criminal Record" (Devah Pager, 2003):

  • Research Focus: Racial discrimination and criminal record stigmatization in hiring practices.

  • Sample & Location: $350$ real-world employers in Milwaukee, WI.

  • Experimental Methodology: Submission of $350$ identical job applications, manipulating race and criminal record status.

  • Factorial Design: $2 \times 2$ design featuring two independent factors:

  • Factor 1: Race (Black vs. White)

  • Factor 2: Criminal Record (Felony Conviction vs. No Felony Conviction)

  • Four Audit Conditions:

  • Black Applicant with No Felony Conviction

  • Black Applicant with Felony Conviction

  • White Applicant with No Felony Conviction

  • White Applicant with Felony Conviction

Quasi-Experimental and Pre-Experimental Designs

• Classification Criteria:

  • Quasi-Experimental Designs: Feature baseline comparisons or control groups but lack Random Assignment.

  • Pre-Experimental Designs: Lack both Random Assignment and control groups (or lack pre-tests).

• Nonequivalent Control Group Design (Quasi-Experimental):

  • Group 1: Non-Random Assignment | Pre-Test: Yes | Intervention: Yes | Post-Test: Yes

  • Group 2: Non-Random Assignment | Pre-Test: Yes | Intervention: No | Post-Test: Yes

• Static Group Comparison / Two-Group Post-Test Only (Pre-Experimental):

  • Group 1: Non-Random Assignment | Pre-Test: No | Intervention: Yes | Post-Test: Yes

  • Group 2: Non-Random Assignment | Pre-Test: No | Intervention: No | Post-Test: Yes

• One-Group Pre/Post-Test Design (Pre-Experimental):

  • Group 1: No Random Assignment | Pre-Test: Yes | Intervention: Yes | Post-Test: Yes (No Control Group)

• One-Shot Case Study (Pre-Experimental):

  • Group 1: No Random Assignment | Pre-Test: No | Intervention: Yes | Post-Test: Yes (No Control Group)

• Methodological Vulnerabilities: Removing key classical elements (random assignment, control groups, pre-tests) increases susceptibility to internal/external validity threats and weakens claims of nomothetic causality.

Observational and Field Demonstrations of Social Dynamics

• "A Class Divided" Demonstration (Jane Elliott, 1968, 1984 Iowa):

  • Theoretical Framework: Labeling Theory.

  • Research Focus: Investigated whether institutionalized and cultural expectations hinder academic performance and drive discriminatory interactions.

  • Experimental Logic Applied:

  • Independent Variable / Stimulus: Arbitrary categorization based on eye color (blue-eyed vs. brown-eyed students).

  • Dependent Variable: Academic performance and behavioral interactions among students.

  • Pre-Test and Post-Test Logic: Evaluated performance before and after applying societal labeling.

• Social Status and Horn-Honking Behavior (Doob and Gross, 1968):

  • Research Focus: Naturalistic field experiment examining the effect of perceived social status cues on driving aggressiveness (horn-honking latency and frequency) in real-world traffic scenarios.