Quantitative Research Designs: Experiments, Surveys & Single-Subject Approaches

Positioning Quantitative Designs within the Research Process

• Research progression in class:
  • Identified a social‐work problem ➜ specified & operationalized variables ➜ crafted a directional hypothesis.
  • Next logical step: pick a quantitative design able to test that hypothesis and reveal the nature (and maybe the direction) of relationships between variables.
• Paradigmatic fit:
  • Most quantitative plans align with positivist or clinical paradigms.
  • Serve descriptive, explanatory, or evaluative objectives.
• Instructor’s caution:
  • Thousands of design variants exist; lecture narrows to those most common in social‐work proposals (surveys & group/ single‐case experiments).

Three Broad Families of Quantitative Designs Covered

• Experimental / Quasi‐experimental / Non‐experimental (group designs)
• Survey (cross‐sectional) designs
• Single‐subject (single‐case/ functional behavioral assessment) designs

Experimental Designs (True Experiments)

• Also called group designs: always involve at least two groups.
  • Experimental group (EG) receives the intervention (independent variable, $X$).
  • Control group (CG) receives no intervention (may get placebo). All other conditions held constant.
• Random sampling (RS) + Random assignment (RA)
  • RS: every element of target population has equal selection probability.
  • RA: once sampled, each participant has equal probability of ending in EG or CG.
• Why RS + RA matter
  • Create equivalent groups ➜ rule out alternative explanations.
  • Enable statements of causality (cause ≠ correlation).
• Key researcher actions = manipulation of $X$ and measurement of $Y$ (dependent variable).
• Linguistic precision: unless all experimental conditions are met, use “predicts,” “correlates,” “relates,” “impacts,” not “causes.”

Classic Pretest–Post-test Control-Group Design

• Scientific notation: $R<em>s \; EG: \; O</em>1 \; X \; O<em>2$ vs. $R</em>s \; CG: \; O<em>1 \; - \; O</em>2$
  • $R<em>s$ = random sample; $R</em>a$ (often capital R) appears before group rows to denote RA.
  • $O<em>1$ = pre-test measure of $Y$; $O</em>2$ = identical post-test.
• Latino mental-health example:
  1. Obtain county list, select every 5th Latino adult (RS).
  2. RA into EG (church-based culturally tailored intervention) vs CG (no intervention).
  3. Pre-test: “How many MH appointments scheduled/attended in last 30 days?”
  4. Deliver intervention only to EG.
  5. One month later post-test same question.
  6. Compare change scores EG vs CG; if statistically significant ➜ infer intervention effect, possibly causality.
• Researcher freedoms: dosage studies (e.g., EG-A = 1-hr, EG-B = 4-hr), timing, setting.

Quasi-Experimental Designs

• Mirror true experiments in procedure/ measurement except they lack RS and/or RA.
• Groups called nonequivalent comparison group (NCG) vs experimental group.
• Reasons for no RS/RA:
  • Practical constraints (no county-wide list; rely on a church convenience sample).
  • Ethical issues.
• Consequences:
  • Cannot claim causation; stick to terms like “associated with,” “related to.”
  • Higher risk of group differences affecting outcomes.
• Notation example (Nonequivalent Comparison Groups Design):
  $EG: \; O<em>1 \; X \; O</em>2$
  $NCG: \; O<em>1 \; - \; O</em>2$
• Same Latino access scenario possible, but participants self-enroll; assignment based on availability.

Internal vs External Validity

• Internal validity (IV)
  • Pertains only to true experiments because it deals with causal inference.
  • 10 classical threats (history, maturation, testing, instrumentation, regression, selection, mortality, interaction effects, etc.).
  • Example of history: ICE raids during immigrant PTSD study confound ACE–PTSD link.
• External validity (EV)
  • Applies to all designs.
  • Concerns generalizability across populations, settings, times.
  • Threats: sample characteristics, setting uniqueness, reactivity to assessment, etc.

Survey (Cross-Sectional) Designs

• Purpose: “temperature check” of attitudes, beliefs, behaviors at one point in time.
• Typical components:
  • Multiple standardized scales (e.g., Beck Depression Inventory, GAD-7, ACEs).
  • Demographic items (race/ ethnicity wording decisions: single vs multiple boxes, write-in).
  • Likert-type response sets (e.g., $1=\text{Strongly Agree} \;\dots\; 5=\text{Strongly Disagree}$).
• Administration modes: in-person, online (Qualtrics), phone.
• Cynthia’s example: flyer in Spanish-speaking church ➜ parents’ opinions on MH & willingness to seek services.
• Strengths:
  • Cost-effective, scalable, fast data collection.
  • Large $n$ improves statistical power.
  • Many items already numeric ➜ simpler analyses.
• Challenges:
  • Biases: social desirability, nonresponse (~70 % may ignore), self-selection.
  • Survey fatigue (end-of-semester deluge at CSUSB).
  • Question wording effects.
  • Limited generalizability without RS.
  • Online risks: bots, data breaches.
• Mitigation strategies: pilot testing, incentives, CAPTCHA, secure servers.
• Likert illustration (Academic Detailing item): respondents tick 1–5 ➜ researcher codes numeric value for statistics.

Single-Subject (Single-Case, A–B, Functional Behavioral Assessment) Designs

• Unit of analysis = one client, family, group, organization, or community.
• Goal: evaluate whether a specific intervention changes a specific behavior for that unique entity.
• Typical structure:
  • A-phase (baseline): repeated measurement of $DV$.
  • B-phase (intervention): introduce $X$; continue measuring $DV$.
  • Variants: A-B-A, A-B-A-B to verify replicability of effect.
• Meditation–stress example:
  1. A (2 weeks): nightly 8 pm rating $1\dots10$ of daily stress.
  2. B (2 weeks): add 15-min guided meditation each morning + continue nightly ratings.
  3. Analyze trend graphs for change & steady state (stable pattern in DV).
• Outcome patterns:
  • Stable increase / decrease / flatline = steady state.
  • Cyclical plateau (e.g., stress spikes every custody week).
  • Unstable/no pattern ➜ intervention likely ineffective or other variables at play.
• Pros & cons:
  • Highly tailored, clinically practical, immediate feedback for practitioner & client.
  • Very low external validity—cannot generalize beyond case.
  • Still subject to ethical guidelines (informed consent, confidentiality).

Practical & Ethical Takeaways

• Always match design to:
  • Research question type (descriptive vs causal).
  • Available resources (sampling frames, time, money, participant pool).
  • Ethical constraints (fair access, burden on participants).
• Language discipline: “cause” only for rigorous experiments; otherwise use correlational terminology.
• Limitations inevitable; explicitly discuss them (internal threats, sampling bias, etc.).
• Next‐step in course: Sampling & Measurement (how to operationalize RS/RA, choose scales, ensure reliability & validity).
• Ongoing assignment reminder: Annotated Bibliography due; use today’s design distinctions to critique methods in your sources.