Sampling Techniques, Data Collection, and Experimental Research Designs

Fundamentals of Sampling

  • Population (Universe): A well-specified, identifiable group (finite or infinite; real or imaginative). A descriptive measure computed from the entire population is a parameter.

  • Sample: A smaller subset drawn from a population to draw inferences about the whole. A descriptive measure computed from a sample is a statistic.

  • Sampling Frame: The actual operational list or database from which sample units are selected. Incomplete frames introduce systematic selection bias.

  • Confidence Interval (CI\text{CI}): The range of values around the sample statistic that is estimated to contain the true population parameter (e.g., a 95%95\% CI\text{CI} signifies 95%95\% confidence that the true population mean falls within the stated boundaries).

  • Significance Level (α\alpha): The threshold probability of rejecting a true null hypothesis (Type I error), routinely established at α=0.05\alpha = 0.05.

Sampling Hierarchy: Theoretical Population to Sample

Probability Sampling Methods

Probability sampling guarantees that every unit in the target population has a known, nonzero selection probability, enabling inferential statistical analysis and minimizing investigator bias.

  • Simple Random Sampling (SRS\text{SRS}): Every combination of nn units holds an equal probability of selection from a complete sampling frame (e.g., lottery/fishbowl method, random number generators).

  • Stratified Sampling: Divides a heterogeneous population into mutually exclusive, homogeneous strata (e.g., gender, geographical region) before conducting random selection within each stratum (proportionately or equally) to safeguard subgroup representation and precision.

  • Cluster Sampling: Involves grouping the population into clusters (typically geographical units), randomly sampling clusters, and assessing units within those clusters. Logistically efficient for dispersed populations, but introduces intra-cluster correlation.

  • Multistage Sampling: Integrates cluster and stratification techniques over multiple successive sampling stages.

  • Probability Proportional to Size (PPS\text{PPS}): Assigns higher selection probabilities to larger sampling units relative to their measured size.

Non-Probability Sampling Methods

Non-probability designs select units through non-random criteria; probability of inclusion cannot be calculated, restricting parameter estimation.

  • Accidental / Convenience Sampling: Relies on readily accessible or volunteering individuals; rapid and cost-effective but carries extreme risk of selection bias.

  • Judgmental / Purposive Sampling: Relies on the researcher's professional judgment to intentionally select participants possessing specific characteristics required for in-depth inquiry.

  • Quota Sampling: Segments the population into specific strata and fills fixed non-random quotas from each stratum.

  • Systematic Sampling: Selects every k-thk\text{-th} individual from a list. Functions as probability sampling if the source list is randomized, but introduces bias if cyclical patterns exist.

  • Snowball Sampling: Identifies initial key informants who refer subsequent participants; used primarily for hidden, stigmatized, or hard-to-reach populations.

  • Saturation Sampling: Qualitative strategy where sampling concludes once subsequent data collection yields no novel insights or themes.

  • Dense Sampling: Involves sampling nearly all members of a narrowly defined setting or subgroup for comprehensive qualitative examination.

Data Collection Methods: Primary and Secondary

Data collection is categorized into primary (firsthand collection by the researcher) and secondary (reanalysis of existing records).

  • Primary Data Sources: Experimental investigations, surveys, self-administered questionnaires, schedules, direct observational studies, and individual/group interviews.

  • Secondary Data Sources: Systematic literature reviews, meta-analyses, governmental/public repositories, corporate and NGO archives, historical documents, mass media, and curated open-data repositories.

Survey, Questionnaire, and Schedule Comparison

Feature

Survey

Questionnaire

Schedule

Definition

Comprehensive method of gathering information from a target sample

Pre-structured set of written questions answered by the participant

Pre-structured question protocol read and completed by the researcher

Mode of Administration

Written, oral, face-to-face, or digital

Self-administered by respondents

Administered directly by an interviewer

Flexibility

High (integrates interviews, forms, observation)

Low (fixed, predetermined questions)

Moderate (interviewer can probe and clarify)

Primary Objective

Broad demographic or attitudinal assessment

Rapid, standardized data gathering

High accuracy; accommodates illiterate/less-educated groups

Example

National opinion poll

Standardized stress questionnaire

In-person household census interview

Observational and Interview Methods

Observational Research
  • Naturalistic Observation: Passive recording of behavior in its authentic context without environmental intervention.

  • Controlled Observation: Systematic recording within an artificially arranged laboratory or standardized setting.

  • Participant Observation: The investigator enters the social setting, participating directly in group activities.

  • Non-Participant Observation: The investigator remains an unobtrusive external recorder.

Qualitative Interviews
  • Structured: Uniform questions delivered in identical sequence; limits investigator flexibility but ensures strict standardization.

  • Semi-Structured: Utilizes a core interview guide while permitting iterative probes, spontaneous clarification, and altered questioning sequences.

  • Unstructured: Open, conversational inquiry into poorly understood or emerging phenomena.

  • Phases: Preparation (objectives, guide design, ethical clearances), Conducting (rapport building, active listening, neutral probing, recording), and Post-Interview (verbatim transcription, reflexive memoing, member checking, inductive coding).

Experimental Research Designs

Experimental methodology isolates cause-and-effect relationships by manipulating independent variables (IVs\text{IVs}), measuring dependent variables (DVs\text{DVs}), and holding extraneous variables constant.

  • Foundations: Developed across the late 19th and early 20th centuries, formalized through advancements by Fisher, Campbell, and Cook.

  • Core Principles: Deliberate manipulation of IVs\text{IVs}, rigorous control of extraneous confounds, implementation of pretests and posttests, and inclusion of comparison control groups.

  • Random Assignment: Allocates participants to conditions independently and with equal probability (e.g., coin toss, computer algorithms, block randomization). Unlike random sampling (which determines sample selection), random assignment determines condition placement.

  • Block Randomization: Sequences treatment conditions such that each condition appears once in randomized order per block before repeating, ensuring equal group sizes across time.

Hypothetical Results Comparing Treatment, No-Treatment, and Placebo Conditions
  • Control and Placebo Effects: Placebos are simulated interventions lacking active therapeutic elements. The placebo effect denotes measurable psychological or physiological improvement stemming from participant expectancy (demonstrated in psychological disorders and physiological conditions like asthma, ulcers, and warts as identified by Shapiro & Shapiro, 1999).

  • Sham Surgery Evidence: J. Bruce Moseley and colleagues (2002) conducted a controlled trial on osteoarthritis of the knee where control subjects received skin incisions and sedation without internal debridement or lavage; sham surgery matched actual surgical intervention in reducing pain and improving function.

Between-Subjects vs. Within-Subjects Designs

Between-Group vs. Within-Group Experimental Design

Feature

Between-Subjects (Between-Group)

Within-Subjects (Within-Group)

Condition Exposure

Each participant experiences only one experimental condition

Each participant completes all experimental conditions

Extraneous Variables

Controlled through random assignment across groups

Fully controlled; participants serve as their own controls

Statistical Power

Lower; subject to individual participant differences

Higher statistical power; reduces error variance

Sample Size

Requires larger total participant pools

Economical; requires fewer total participants

Primary Confound

Individual differences between experimental arms

Carryover Effects: practice, fatigue, and context effects

Control Techniques

Matching, random assignment, block randomization

Counterbalancing (e.g., Latin squares, reversed sequences) and rest intervals

Best Application

Irreversible interventions (e.g., surgical or therapeutic trials)

Reversible cognitive or perceptual tasks where fatigue can be minimized

Program Evaluation Research

Program evaluation systematically applies research methodologies to assess the conceptualization, execution, and effectiveness of social interventions.

Five Progressive Domains
  1. Needs Assessment: Analyzes the presence, severity, and target demographic of a social problem.

  2. Program Theory & Design: Evaluates the conceptual framework, logic model, and feasibility of the planned strategy.

  3. Process / Implementation: Evaluates fidelity of service delivery, operational execution, and resource allocation.

  4. Impact / Outcomes: Determines quantifiable net changes directly attributable to the intervention.

  5. Efficiency / Cost-Effectiveness: Measures economic efficiency by evaluating benefits produced relative to fiscal expenditure.

Correlational Research

Correlational studies quantitatively measure the strength and direction of the statistical association between two or more variables without experimental manipulation or causal attribution.

  • Correlation Coefficient (rr): Standardized metric spanning −1.00-1.00 to +1.00+1.00.

    • Positive Correlation (r>0r > 0): Variables change in the same direction concurrently.

    • Negative Correlation (r<0r < 0): Variables exhibit an inverse relationship.

    • Zero Correlation (r=0r = 0): Indicates an absence of linear association.

  • Primary Applications: Applied when variables cannot be ethically or practically manipulated (e.g., tracking the impact of psychological trauma), during naturalistic baseline observations (e.g., patient wait-time relative to hospital specialty), and for predictive modeling or hypothesis generation.

Questions & Discussion

  • Did each student have an equal chance of being selected in the fishbowl sampling activity? Why or why not?

    • Yes, provided that every student submitted exactly one chit of identical size into the container and the slips were mixed thoroughly, ensuring uniform selection probability.

  • How is this different from asking for volunteers?

    • Volunteer recruitment introduces self-selection bias based on individual motivation and availability, whereas randomized physical drawing preserves objective probability.

  • What makes this a type of Probability Sampling?

    • The selection probability of every population unit is mathematically known, non-zero, and equal.

  • What are the advantages and limitations of this method in real research?

    • While conceptually straightforward and unbiased for small, contained populations, it becomes logistically unfeasible, slow, and expensive when applied to large, geographically dispersed populations lacking a consolidated physical roster.