Sampling

Foundations of Sampling in Social Research

• Rationale for Sampling:

  • Complete population enumeration is generally impossible due to prohibitive logistical costs and practical time constraints.

  • Studying well-designed subunits enables valid statistical inferences regarding the characteristics of the broader population.

• Fundamental Terminology:

  • Population: The complete set of all relevant data, units, or cases of analytical interest.

  • Sample: A smaller subset selected directly from the population.

  • Parameter: A numerical property or variable summarizing a characteristic of an entire population.

  • Statistic: An empirical estimate of a population parameter derived from observations within a sample.

  • Representativeness: The degree to which the characteristics of a sample accurately mirror the parameters of the target population, which is necessary for valid generalizability.

• Initial Research Decisions:

  • Definition of the Population: Establishing concrete boundaries, eligibility criteria, and unit definitions.

  • Sample Design: Selecting between probability and non-probability sampling strategies.

  • Size of the Sample: Determining the numerical volume of observations needed based on analytic goals and acceptable error margins.

The Sampling Frame and the Sampling Process

• Sampling Frame Definition:

  • An operational, comprehensive listing of all sampling units or individual elements within the target population from which the actual sample is drawn.

  • It must contain every single element of the population of interest to prevent coverage errors.

• The Four-Stage Sampling Discrepancy Framework:

  • Target Population: The abstract conceptual universe of cases under study (what a researcher would like to talk about).

  • Sampling Frame: The actual operational list or roster available for selection.

  • Sampling Process: The specific procedural mechanism applied to extract units from the frame into the empirical dataset.

  • Empirical Sample: The final group of selected units from which data is gathered (what is actually observed in the data).

Historical Case Study: The 1936 Presidential Election Poll Failure

• Context and Polling Prediction:

  • In the 1936 United States Presidential Election, The Literary Digest conducted a massive straw poll of over 2 million participants.

  • Poll Prediction:

  • Alf Landon: $57\%$

  • Franklin D. Roosevelt: $43\%$

  • Actual Election Outcome:

  • Alf Landon: $38\%$

  • Franklin D. Roosevelt: $62\%$

  • Electoral Vote Outcome:

  • Franklin D. Roosevelt won an overwhelming landslide across 46 states ($523$ electoral votes).

  • Alf Landon won only 2 states: Maine ($5$ electoral votes) and Vermont ($3$ electoral votes), totaling $8$ electoral votes.

• Methodological Diagnostic:

  • Sampling Frame Bias: The magazine drew contact lists from telephone directories, automobile registrations, and its own subscriber rolls. During the Great Depression, these lists systematically over-represented higher-income households leaning Republican.

  • Nonresponse Bias: Responses were voluntarily returned by mail, creating a self-selection bias where disenfranchised, dissatisfied Landon supporters were substantially more motivated to respond.

  • Core Principle: An enormous sample size cannot compensate for structural sampling frame bias; representativeness is mathematically paramount.

Probability Sampling Designs

• Core Tenets of Probability Sampling:

  • Every unit in the population has a known, non-zero mathematical probability of selection.

  • Enables formal statistical inferences about unknown population parameters.

  • Objective, probabilistic randomization minimizes systematic sampling bias.

  • Requires a complete, well-defined sampling frame, which can be logistically challenging to assemble.

• Simple Random Sampling (SRS):

  • Every unit in the sampling frame has an equal probability of selection.

  • Implementation: Assign each unit an identification number from $1$ to $n$, then select cases using an objective random number generator, computer script, or lottery device until reaching the target sample size.

• Stratified Sampling:

  • Divides the sampling frame into mutually exclusive, homogeneous subsamples known as "strata" based on specific demographic or organizational variables.

  • Simple random sampling is subsequently executed independently within each stratum, ensuring exact proportional representation of smaller subgroups.

  • Application 1: Age Strata Sample ($n = 1{,}000$):

  • Under 18: $3\%$ of population $\rightarrow 30$ individuals

  • 18–24: $72\%$ of population $\rightarrow 720$ individuals

  • 25–39: $21\%$ of population $\rightarrow 210$ individuals

  • 40 or Older: $4\%$ of population $\rightarrow 40$ individuals

• Application 2: Institutional Enrollment Stratification (Stony Brook University, $N = 25{,}865$, Sample Target $n = 1{,}000$):

  • College of Arts & Sciences: $12{,}061$ students ($46.6\%$) $\rightarrow 466$ units

  • College of Business: $1{,}882$ students ($7.3\%$) $\rightarrow 73$ units

  • School of Communication & Jrn: $289$ students ($1.1\%$) $\rightarrow 11$ units

  • School of Dental Medicine: $204$ students ($0.8\%$) $\rightarrow 8$ units

  • College of Engineering & App Sci: $5{,}302$ students ($20.5\%$) $\rightarrow 205$ units

  • Family Medicine: $49$ students ($0.2\%$) $\rightarrow 2$ units

  • School of Health Professions: $1{,}151$ students ($4.5\%$) $\rightarrow 45$ units

  • Health Sciences Center: $158$ students ($0.6\%$) $\rightarrow 6$ units

  • School of Marine & Atmos Sci: $771$ students ($3.0\%$) $\rightarrow 30$ units

  • School of Medicine: $710$ students ($2.7\%$) $\rightarrow 27$ units

  • School of Nursing: $970$ students ($3.8\%$) $\rightarrow 38$ units

  • School of Professional Dev: $1{,}240$ students ($4.8\%$) $\rightarrow 48$ units

  • Provostial Area/Other: $244$ students ($0.9\%$) $\rightarrow 9$ units

  • School of Social Welfare: $628$ students ($2.4\%$) $\rightarrow 24$ units

  • Southampton Arts: $206$ students ($0.8\%$) $\rightarrow 8$ units

  • Total Stratified Sample Allocation: $1{,}000$ units

• Cluster Sampling (Multi-Stage Area Sampling):

  • Primary method for generating face-to-face probability samples across large, geographically dispersed populations without an existing master directory.

  • Successively samples progressively smaller geographic units or spatial clusters.

  • Multi-Stage Geographic Hierarchy:

  • Nation: Broadest geographical territory.

  • Primary Sampling Unit (PSU): Regional county or metropolitan zone.

  • Sample Place: Individual town or distinct municipal boundary.

  • Chunk: Specific community enclave or sub-neighborhood.

  • Segment: Specified city street block bounded by roads (e.g., area framed by Hudson Ave., Main St., Levy St., and Washington St.).

  • Dwelling Unit: The individual home or physical residence targeted for interview.

  • Trade-Offs: Cost-effective for broad face-to-face field logistics, but administratively complex and introduces higher sampling errors compared to SRS.

• Systematic Sampling:

  • Uses a standardized procedural selection interval to choose cases from a list.

  • Sampling Interval Calculation:     $k = \frac{\text{Population Size}}{\text{Sample Size}}$

  • Example: For a population of $10{,}000$ and a target sample of $100$:     $k = \frac{10{,}000}{100} = 100$     The researcher selects every $100^{\text{th}}$ element on the roster.

  • Trade-Offs: Highly efficient, practical proxy for simple random sampling; however, it risks systematic bias if the list has hidden periodic cycles coinciding with the sampling interval.

Nonprobability Sampling Designs

• Characteristics of Nonprobability Sampling:

  • Individual units lack a calculable, known probability of inclusion.

  • Cannot produce mathematically valid generalizations or population parameter inferences.

  • Employed primarily when sampling frames are nonexistent, in preliminary or exploratory investigations, and in qualitative fieldwork.

• Common Nonprobability Types:

  • Convenience Sampling: Selects easily accessible, readily available units (e.g., intercepting people walking past a street corner).

  • Snowball (Chain-Referral) Sampling: Existing participants recruit or identify subsequent participants through social network ties. Highly effective for reaching hidden, marginalized, or stigmatized groups (e.g., undocumented immigrants, commercial sex workers, transient populations).

  • Purposive (Judgmental) Sampling: Cases are intentionally hand-selected according to the professional subjective assessment of the researcher regarding what constitutes a "typical" or informative case.

  • Deviant Case Sampling: Explicitly seeks out atypical, highly unusual cases that violate dominant patterns or normative expectations to explore boundary conditions.

  • Quota Sampling: Assembles a sample matching pre-specified population demographic distributions using non-random convenience methods.

  • Demographic Quota Example:

    • Target Population ($N = 1{,}000$):

    • $70\%$ White, non-Hispanic ($n = 700$)

    • $20\%$ African American ($n = 200$)

    • $10\%$ Latinx ($n = 100$)

    • Quota Sample ($n = 100$):

    • Exactly $70$ White, non-Hispanic individuals

    • Exactly $20$ African American individuals

    • Exactly $10$ Latinx individuals

  • Distinction between Stratified and Quota Sampling: Stratified sampling relies on strict random probability selection within each stratum, whereas quota sampling uses non-probabilistic convenience recruitment to fulfill demographic targets.

Sample Size Determination and Statistical Power

• Determining Sample Size:

  • Sample size requirements depend on research objectives, specific analytical tools, underlying population variance, and the acceptable standard error.

  • Qualitative inquiries aim for data and theoretical saturation (the point at which new observations cease to provide novel conceptual insights).

  • Basic quantitative inquiries rely on a general baseline rule of thumb requiring at least $20$ observations for simple statistical operations.

• Power Analysis and Standard Error Mathematical Estimation:

  • Sample size can be derived by balancing acceptable standard error ($\text{SE}$) against population variance:     $\text{Sample Size} = \frac{\text{Standard Deviation}^2}{\text{Standard Error}^2}$     $n = \frac{s^2}{\text{SE}^2}$

  • Increasing sample size reduces standard error and improves precision, though marginal returns diminish as sample size increases.

The Normal Distribution and the Central Limit Theorem

• Properties of the Normal Distribution:

  • Unimodal, symmetric, bell-shaped continuous distribution.

  • The central point satisfies:     $\text{Mean} = \text{Median} = \text{Mode}$

• Central Limit Theorem (CLT):

  • As the size of an independent random sample ($n$) increases, the sampling distribution of the sample mean converges toward a normal distribution, regardless of the underlying population's shape.

  • As sample size increases, estimates of the population parameter become increasingly accurate and predictable.

  • This theorem underpins many parametric inferential statistics that assume underlying normality.

• Dice Roll Simulation Evidence:

  • When rolling a pair of fair dice, outcome sums range from $2$ to $12$.

  • At small iterations ($5$ or $10$ rolls), the distribution appears erratic and non-uniform.

  • As rolls increase ($50$, $100$, $1{,}000$, $10{,}000$, up to $1{,}000{,}000$ or $1\text{e}+06$ rolls), the distribution forms a smooth, unimodal, symmetric normal curve centered directly over the expected theoretical mean of $7$.

Confidence Intervals and Inferential Estimation

• Confidence Interval Principles:

  • A defined range of numerical values centered around a sample statistic, within which researchers can state with a predetermined degree of certainty that the true population parameter lies.

  • Finite sample sizes invariably introduce sample-to-sample variability, requiring confidence intervals to quantify uncertainty.

• Empirical Rule and Critical Inferential Thresholds:

  • Across repeated random samples:

  • Approximately $68\%$ of sample means fall within $\pm 1$ standard error ($\pm 1\,\text{SE}$) of the true population mean ($\mu \pm 1\sigma$ covers $68.26\%$; each symmetrical half comprises $34.13\%$).

  • Exactly $95\%$ of all random sample means fall within $\pm 1.96$ standard errors ($\pm 1.96\,\text{SE}$) of the population mean (approximated as $\pm 2\,\text{SE}$ or $\mu \pm 2\sigma$, which encompasses $95.44\%$; the adjacent segment between $1\sigma$ and $2\sigma$ adds $13.59\%$ per tail).

  • Exactly $99\%$ of all random sample means fall within $\pm 2.58$ standard errors ($\pm 2.58\,\text{SE}$) of the population mean ($\mu \pm 3\sigma$ covers $99.74\%$; the segment between $2\sigma$ and $3\sigma$ contributes $2.28\%$ per tail).

Review & Conceptual Assessment Questions

• Question 1: Random selection entails:

  • Options:

  • A. Randomly choosing a case number out your head

  • B. Choosing someone walking by at random

  • C. Using an objective selector (random number generator, technical application, script, "lottery machine", or pulling names out of a hat) to select a case from all possible members of the population.

  • Correct Answer: C. Objective randomizing mechanisms ensure every member of the population retains a known, non-zero probability of inclusion.

• Question 2: What is the difference between Stratified Sampling and Quota Sampling?

  • Options:

  • A. Quota Sampling is a probabilistic method, Stratified Sampling is not.

  • B. Stratified Sampling is a probabilistic method, Quota Sampling is not.

  • Correct Answer: B. Stratified sampling applies formal random lottery mechanisms within homogeneous strata, while quota sampling fills demographic targets using non-probabilistic convenience recruitment.

• Question 3: Which of these sampling methods is a probabilistic method that uses geographical space?

  • Options:

  • A. Convenience

  • B. Cluster

  • C. Stratified

  • D. Systematic

  • Correct Answer: B. Cluster sampling executes multi-stage probabilistic selection across geographic units, from primary sampling units down to individual dwelling units.