Notes on Experimental vs Non-Experimental Methods

Experimental vs Non-Experimental Methods

  • Experimental methodology
    • Systematic, controlled conditions
    • Tests a hypothesis to establish a causal relationship between independent (IV) and dependent variables (DV)
  • Non‑experimental methodology
    • Used when control or ethics prevent an experiment
    • Describes behavior; cannot infer causation
  • Definition: Causation – a relationship where one variable directly influences another.
    • \text{Causation} \equiv \text{direct influence of IV on DV}
    • In contrast, correlation does not imply causation.
  • Types of Non‑Experimental Designs
    • Case Study – intensive study of a single individual, group, or event
    • Risk: Hawthorne effect – participants alter behavior because they know they’re being observed
    • Correlational Study – examines the strength and direction of the relationship between two variables
    • Important: Correlation ≠ Causation – only a controlled experiment can determine cause‑and‑effect
    • Susceptible to the third‑variable problem (an unmeasured variable influences both studied variables)
    • Meta‑analysis – statistical technique that combines results of multiple studies on the same topic to reach a more robust conclusion
  • Feature comparison (Experimental vs Non‑experimental)
    • Control of variables: High (Experimental) vs Low (Non‑experimental)
    • Ability to infer causation: ✅ (Experimental) vs ❌ (Non‑experimental)
    • Typical designs: Lab experiments, field experiments (Experimental) vs Case studies, correlational studies, meta‑analysis, naturalistic observation (Non‑experimental)
  • Naturalistic Observation
    • Observing behavior in its real‑world setting
    • Challenge: limited contextual knowledge can lead to misinterpretation (e.g., observing a school during a temporary COVID‑19 setup)

Designing a Study

  • Formulating the Hypothesis
    • A specific, testable prediction about the relationship between variables
    • Must be falsifiable (capable of being proven wrong)
    • Example: “Students who use the Ultimate Review Packet will score higher on the AP Psychology exam than students who do not use it.”
  • Operational Definitions
    • Precise description of how each variable will be measured or manipulated
    • Enables replication
    • Sleep Study Example:
    • More sleep: ≥8 hours of continuous sleep (tracked with Apple Watch)
    • Less sleep: <8 hours of continuous sleep
    • Performance: Score on the AP Psychology National Exam, reported on a 1–5 scale by the College Board
  • Identifying Variables
    • Confounding variables reduce internal validity; the more control, the fewer confounds, but over‑control can create an inauthentic environment
    • Variable Type → Role → Example (Sleep Study)
    • Independent Variable (IV) – Manipulated cause – Amount of sleep: $S \ge 8\text{h}$ vs $S < 8\text{h}$
    • Dependent Variable (DV) – Measured effect – Exam score
    • Confounding Variables – Uncontrolled factors that may affect DV – Study habits, stress level, overall health

Participants: Population vs. Sample

  • Population – the entire group of interest
  • Sample – a representative subset of the population used in the study
  • Sampling Techniques
    • Sampling bias – systematic error where the sample does not reflect the population, limiting generalizability (the extent findings apply to the broader population)

Experimental vs. Control Groups

  • Experimental group – receives the IV (e.g., review packet)
  • Control (placebo) group – receives a neutral version lacking the critical component of the IV
  • Random Assignment
    • Participants are randomly assigned to experimental or control groups, ensuring equivalent groups and reducing bias
    • Note: Random assignment ≠ random selection (the latter pertains to how participants enter the study)

Quasi‑Experiments

  • Used when random assignment is unethical or impossible (e.g., assigning depression)
  • Lacks true randomization → cannot definitively establish causation
  • Key Principle: Only true experiments with random assignment can reliably infer cause‑and‑effect relationships

Procedures: Blind Designs

  • Single‑blind procedure – participants do not know whether they are in the experimental or control group
    • Reduces social desirability bias and the placebo effect
  • Double‑blind procedure – both participants and researchers are unaware of group assignments
    • Mitigates experimenter bias as well as social desirability bias
  • Biases and effects to be aware of
    • Social desirability bias – Tendency of participants to answer in a way that will be viewed favorably by others
    • Placebo effect – Improvement in a participant’s condition that occurs because they believe they are receiving an active treatment
    • Experimenter bias – Unconscious influence of a researcher’s expectations on the outcome of a study

Measurements: Qualitative vs. Quantitative

  • Qualitative measures
    • Collect non‑numerical data for rich, descriptive insight
    • Example: Structured interviews with open‑ended questions
    • Strength: Captures participants’ thoughts, feelings, and experiences in depth
    • Limitation: Subjective, harder to replicate
    • Blind Type – Single‑blind: Participants only; addresses social desirability, placebo
    • Blind Type – Double‑blind: Participants and researchers; addresses experimenter bias, social desirability
    • Limitation: Subjective, harder to replicate
  • Quantitative measures
    • Gather numerical data amenable to statistical analysis
    • Example: Likert scale (rating agreement from strongly disagree to strongly agree)
    • Strength: Objective, facilitates replication and statistical testing
    • Limitation: May oversimplify complex phenomena
  • Qualitative vs. Quantitative vs. Mixed methods (not explicitly stated, but implied)
  • Data Forms and Typical Tools (summary)
    • Qualitative: Non‑numerical data; Structured interviews, observations; Pros: Rich detail; Cons: Subjective, less reproducible
    • Quantitative: Numerical data; Likert scales, surveys, physiological metrics; Pros: Objective, statistical power; Cons: May miss nuance

Protecting Participants: Ethics & Consent

  • Informed consent
    • Participants receive complete information about the study’s purpose, procedures, risks, and benefits, allowing a rational, voluntary decision to take part
  • Informed assent
    • Used when participants cannot legally give full consent (e.g., minors); requires parent/guardian permission plus the participant’s agreement
  • Ethical research practices
    • Create a trusting environment, ensure no harm, and aim for societal benefit
  • Measurement & Data Considerations
    • Informed consent – Process by which participants are fully briefed on a study’s details and voluntarily agree to participate
  • Typical Tools, Pros, and Cons (conceptual overview)
    • Qualitative data – rich, context; Pros: depth; Cons: subjectivity and reproducibility concerns
    • Quantitative data – numerical; Pros: objectivity and power; Cons: may oversimplify

Informed Assent and Historical Milestones

  • Informed assent – Consent process for individuals (often minors) who cannot legally consent on their own; requires guardian approval
  • Historical Milestones
    • Researchers must debrief participants after the study, explaining the true purpose and any deception used

Conclusions: Peer Review & Replication

  • Peer review – Independent experts evaluate a study’s methodology, data, and conclusions before publication
  • Replication – Independent researchers repeat the study to verify findings
  • Together, peer review and replication uphold scientific standards, ensuring findings are reliable, valid, and generalizable
  • Timeline highlights (Significance)
    • 1892 – American Psychological Association (APA) founded; Governing body for psychology research
    • 1947 – APA’s first Ethical Committee established; research standards
    • 1974 – Creation of Institutional Review Board (IRB); formal protection of human participants; Institutional Animal Care and Use Committee (IACUC) oversees ethical treatment of animal subjects

Key Terms Glossary (quick reference)

  • IV: Independent Variable – manipulated cause
  • DV: Dependent Variable – measured effect
  • Confounding Variable: Uncontrolled factor that can influence DV
  • Hawthorne Effect: Participants alter behavior due to being observed
  • Placebo Effect: Improvement due to belief of receiving treatment
  • Random Assignment: Randomly assigning participants to groups to create equivalent groups
  • Random Selection: How participants enter the study; different from random assignment
  • Naturalistic Observation: Observing behavior in real settings without manipulation
  • Meta-analysis: Statistical synthesis of multiple studies
  • Quasi‑Experiment: Non‑randomized design that cannot definitively infer causation
  • Blind Design: Single‑blind or double‑blind procedures to reduce bias
  • Informed Consent: Legal and ethical authorization to participate
  • Informed Assent: Assent from participants unable to provide legal consent (with guardian approval)
  • Debriefing: Post‑study explanation of true purpose and any deception used
  • IRB/IACUC: Regulatory bodies protecting human and animal subjects in research