Comprehensive Guide to Sampling Methods, Survey Design, and Sample Size Estimation

Historical Context of Sampling

• The origin of modern political sampling dates back to 1920, initiated by the Literary Digest, a U.S. news magazine published between 1890 and 1938.

• The Literary Digest successfully predicted the U.S. presidential election outcomes in 1920, 1924, 1928, and 1932 using large-scale reader polling.

• In 1936, the magazine's sampling methodology failed catastrophically:

  • The poll utilized a massive sample size of $10,000,000$ individuals drawn from government registries of automobile owners and telephone directories.

  • The survey predicted that Alf Landon would defeat Franklin D. Roosevelt by a significant margin.

  • Instead, Franklin D. Roosevelt won the election in a historic landslide.

  • Cause of Failure: Severe sampling frame bias. During the Great Depression in 1936, automobile and telephone ownership was restricted primarily to wealthy individuals who overwhelmingly favored Alf Landon, rendering the sampling frame unrepresentative of the broader voting population.

Fundamental Concepts and Definitions

• Sample: A specially selected subset or portion of a larger population chosen to represent the population as a whole.

• Sampling: The specific procedure or process by which a portion of a population or universe is selected as representative of that entire group.

• Population (Universe) / Target Population:

  • The complete collection of all individuals, families, groups, organizations, or events under consideration.

  • The broader group to which a researcher intends to generalize research findings (e.g., all adult residents of Myanmar aged 65 years or older).

• Parameter: A summary numerical measure calculated to describe a characteristic of an entire population.

• Statistic: A summary numerical measure calculated from sample data to describe a characteristic of that sample.

• Census: A complete enumeration and data collection gathered from every single member of a target population.

  • Comparison Example: A Gallup poll constructs representative national samples of adult populations using approximately $1,500$ respondents. This sample size achieves a $95\%$ confidence level with a margin of error within $\pm 3\%$ percentage points, proving far more efficient than conducting a census.

• Sampling Unit / Element / Unit of Analysis:

  • Element: Each individual member of a target population (e.g., a child under 5 years of age).

  • Sampling Unit: The specific unit about which information is collected during sampling.

  • Unit of Analysis: The fundamental entity that provides the basis of statistical analysis (e.g., households when assessing any injury occurring within a household over the preceding three months).

• Sampling Frame: The actual, comprehensive list or registry of sampling units from which the sample (or a stage of the sample) is directly drawn.

• Sample Design: The explicit set of rules, protocols, or procedures specifying how elements are selected from the frame (classified into probability or non-probability methods).

• Sample Size ($n$): The total number of individual elements or units contained within the obtained sample.

Reasons for Drawing a Sample

• Time Efficiency: Conducting a sample survey requires significantly less time than gathering data from an entire population via a census.

• Cost Reduction: Administering a survey to a sample involves dramatically lower operational and resource costs compared to a full census.

• Practicality: Sampling avoids the cumbersome administrative complexities and logistical challenges associated with locating and interviewing every member of a targeted population.

Classification of Sampling Methods

• Sampling techniques are broadly divided into two main categories: Probability Sampling and Non-Probability Sampling.

Probability Sampling Techniques

• Definition: Sampling procedures in which every individual unit in the population has a known, non-zero probability or chance of being selected into the sample.

• Simple Random Sampling:

  • Every individual item or element in the sampling frame has an equal and independent chance of selection.

  • Selection protocols can be executed with replacement or without replacement.

  • Samples are generated using formal tools such as standardized tables of random numbers or computer-based random number generators.

  • Simple random samples are inherently unbiased and yield representative estimates of the population on average.

• Systematic Sampling:

  • A method involving a systematic selection interval ($k$) paired with a initial random start.

  • Procedure:

  1. Determine the total population size ($N$) and required sample size ($n$).

  2. Calculate the sampling interval: $k = \frac{N}{n}$.

  3. Randomly select an initial individual unit from the first group of size $k$

  4. Select every $k$\text{-th} individual sequentially across the sampling frame from that starting point onward.

  • Filarial Survey Example: To sample $10\%$ of a town's total population of $N = 5,000$, the target sample size is $n = 500$, giving a sampling interval of $k = 10$. If house number $3$ is randomly selected from the first group ($1$ to $10$), the subsequent selected units will be $3, 13, 23, 33, 43,\dots$ until $n = 500$ units are collected.

• Stratified Random Sampling:

  • Involves dividing the target population into distinct, mutually exclusive subgroups called strata based on specific shared characteristics (e.g., age, sex, socioeconomic status, or religion).

  • An independent simple random sample is subsequently drawn from within each individual stratum, and the resulting samples are combined into a final single sample.

  • Utility: Ensures adequate representation of key demographic characteristics and prevents rare subgroups from being omitted. For instance, if a rare population subgroup constitutes only $3\%$ of the total population (e.g., African voodoo healers), standard simple random sampling might omit them entirely, whereas stratified sampling guarantees their inclusion.

  • Proportional Example: If a target population consists of $30\%$ males and $70\%$ females, a stratified sample of $100$ total individuals is generated by randomly choosing $30$ males from the male stratum and $70$ females from the female stratum, matching the target population proportions exactly.

• Cluster Sampling:

  • A sampling strategy where the primary sampling unit selected is a group or "cluster" of elements (e.g., city blocks, families, or geographic regions) rather than an isolated individual.

  • Indications for Use:

  • When a comprehensive sampling frame listing individual elements is completely unavailable or excessively expensive to construct.

  • When the economic or logistical costs of reaching geographically dispersed individual elements are prohibitively high (e.g., interviewing automobile mechanics spread across Myanmar, where building a national registry and traveling to individually selected mechanics nationwide would require an unfeasible budget).

  • Procedure: Large clusters that are representative of the overall population (such as cities) are identified. A simple random sample of these clusters is selected (e.g., choosing $10$ cities instead of $200$), lists of target elements within those chosen clusters are obtained, and elements are sampled within the selected clusters.

  • Trade-off: Cluster sampling is substantially less expensive and logistically easier, but it yields lower statistical precision and accuracy compared to simple random sampling due to intra-cluster similarity.

• Multi-stage Sampling:

  • A complex probability technique involving sequential sampling choices across multiple organizational or geographical levels (e.g., sampling regions, then sampling districts within chosen regions, then sampling villages within chosen districts, and finally sampling households within chosen villages).

Non-Probability Sampling Techniques

• Definition: Methods where the probability or chance of any particular population unit being included in the sample is completely unknown, preventing the calculation of sampling error or formal statistical generalization.

• Convenience Sampling:

  • Selection of sample units based solely on ease of access or immediate availability (e.g., "man-in-the-street" interviews or soliciting blood pressure measurements from walking volunteers at public health booths).

  • Limitations: Generalizing findings from convenience samples is invalid because unknown systematic biases are continuously operating.

  • Temporal/Geographic Clustering Example: Interviewing people walking past an office produces severe variable bias depending on time:

  • At 8:00 AM in Tokyo, pedestrians consist almost entirely of employed individuals commuting to work.

  • At 10:00 AM, the proportion of non-working individuals and women rises significantly.

  • At Midnight, the surrounding street population shifts to young socializers.

• Quota Sampling:

  • Entails selecting units haphazardly or by convenience until pre-established sample quotas for specific demographic categories are satisfied.

  • Represents a slight design improvement over basic convenience sampling, but retains systematic selection biases.

  • Methodological Vulnerabilities: Lacks clear objective criteria for identifying which demographic categories are truly key to the research, and lacks objective guidelines for determining exact category sample sizes.

• Purposive / Judgment Sampling:

  • Elements are chosen deliberately based on expert judgment and existing subject matter knowledge regarding the research problem to select cases that are most informative.

  • Applications: Ideal for exploratory research, qualitative study designs, preliminary theoretical development, and pre-testing survey questionnaires.

• Snowball Sampling:

• A chain-referral sampling process where initial interviewed respondents are asked to identify and recommend additional potential subjects from their personal social networks.

• Primary Application: Essential for researching hidden, hard-to-reach, invisible, or marginalized populations (e.g., active drug addicts, undocumented workers, or illicit network participants).

Survey Errors and Non-Sampling Errors

• Non-Sampling Errors: Systematic errors introduced during survey implementation unrelated to statistical sampling variation. These include:

  • Inadequate sampling frame construction (Non-coverage).

  • Non-response from selected eligible participants.

  • Measurement and response errors during data collection.

  • Data processing, coding, and entry errors.

• Four Core Types of Survey Errors:

  • Coverage Error: Occurs when specific segments of the target population are systematically excluded from the sampling frame.

  • Non-Response Error: Occurs when collected responses differ systematically from non-respondents due to an inability to contact selected individuals or direct refusals. Managed through systematic follow-up strategies.

  • Sampling Error: The inherent, unavoidable statistical divergence that occurs naturally from sample to sample due to chance.

  • Measurement Error: Inaccuracies in collected data stemming from poorly designed, ambiguous, or leading survey questions ("Bad Question!"), interviewer bias, or respondent misreporting.

Strategies for Improving Survey Response Rates

• Enhancing overall response rates requires targeted operational interventions designed to reduce both participant refusals and "not-at-home" non-contacts:

• Reducing Participant Refusals:

  • Prior Notification: Contacting prospective respondents in advance via letters, emails, or phone calls before administering the main instrument.

  • Motivating Respondents: Articulating the broader social importance, utility, and relevance of the research topic.

  • Incentives: Offering financial honorariums, token gifts, or lotteries to compensate respondents for their time.

  • Questionnaire Design and Administration: Crafting brief, engaging, clear, and unambiguous questions while maintaining professional administration standards.

  • Follow-Up: Sending reminder postcards, letters, or follow-up phone calls to non-responders.

  • Other Facilitators: Utilizing trained, empathetic interviewers and ensuring strict confidentiality.

• Reducing Not-at-Homes:

  • Callbacks: Implementing repeated, systematically timed contact attempts (varying the day of the week and time of day) to reach selected respondents who were initially absent.

Criteria for Evaluating Survey Worthiness

• Assess the fundamental purpose and scientific objective of the survey.

• Verify whether the survey is grounded in a rigorous probability sampling design.

• Evaluate potential Coverage Error: Check if the sampling frame adequately and accurately encompasses the target population.

• Evaluate potential Non-Response Error: Review non-response rates and verify if standardized follow-up protocols were executed.

• Evaluate potential Measurement Error: Inspect question design to ensure questions are unbiased, clear, and eliciting valid measures.

• Account for Sampling Error: Recognize that random sampling variation always exists and calculate appropriate confidence bounds.

Sample Size Determination and Formulas

• Core Objective: Select an adequate sample size ($n$) to achieve both a desired level of statistical precision and a high level of confidence while minimizing research costs and operational burden.

• Determinants of Sample Size:

  • The specific statistical analysis technique to be employed.

  • The level of precision or acceptable error margin required.

  • Population homogeneity or heterogeneity regarding the variable of interest.

  • Total available financial, human, and time resources.

  • The specific sampling technique utilized (e.g., stratified designs vs. cluster designs).

• Standard Sample Size Formula (Estimation of Proportions):

  • The primary formula for calculating required sample size when estimating population proportions is:

  $n = \frac{z^2 (p q)}{e^2}$

• Formula Parameters:

  • $n$ = Desired sample size.

  • $z$ = Standard normal deviate corresponding to the targeted confidence level (for a $95\%$ confidence level, standard $z = 1.96$).

  • $p$ = The estimated proportion of the target population possessing the specific characteristic of interest. If no prior empirical estimate is available, set $p = 0.50$ ($50\%$) to maximize expected variance and yield the most conservative (largest) sample size estimate.

  • $q$ = Complement of the estimated proportion, calculated as $q = 1 - p$ (or $100 - p$ if expressed as a percentage).

  • $e$ (or $d$) = Acceptable margin of error / absolute precision level (commonly set to $0.05$, representing $\pm 5\%$).

• Standard Example Calculation:

  • Assuming $z = 1.96$ ($95\%$ confidence interval), $p = 0.50$, $q = 0.50$, and precision $e = 0.05$:

    $n = \frac{(1.96)^2 \times (0.50) \times (0.50)}{(0.05)^2}$

    $n = \frac{3.8416 \times 0.25}{0.0025}$

    $n = \frac{0.9604}{0.0025} = 384.16$

- Rounded up to the nearest whole integer, the minimum required sample size is $n = 384$.

• Yamane's Formula (Taro Yamane, 1973):

  • Simplified sample size calculation formula for finite populations:

  $n = \frac{N}{1 + N(e)^2}$

• Formula Parameters:

  • $n$ = Sample size.

  • $N$ = Total finite population size.

  • $e$ = Acceptable margin of error / precision level (set at $0.05$ for a $95\%$ confidence or reliability level).

• Yamane Worked Example:

  • Calculate the sample size ($n$) required for a research population of $N = 37,581$ individuals at a precision error level of $e = 0.05$

    $n = \frac{37,581}{1 + 37,581 \times (0.05)^2}$

    $n = \frac{37,581}{1 + 37,581 \times 0.0025}$

    $n = \frac{37,581}{1 + 93.9525}$

    $n = \frac{37,581}{94.9525} \approx 395.78$

- Rounded up to the nearest integer, the required sample size is $n \approx 396$

Operational Steps Prior to Fieldwork

• Before initiating survey data collection in the field, execute the following operational steps:

  • Work Plan Development: Establish definitive project timelines and map out complete fieldwork logistics.

  • Financing and Budgeting: Secure funding and allocate capital across field operations.

  • Instrument Development: Finalize and format questionnaires or clinical recording sheets.

  • Household Sampling Execution: Draw the exact physical sample of households from the sampling frame.

  • Training Manual Creation: Author comprehensive protocols for standardized data collection.

  • Pilot Testing: Conduct pre-fieldwork testing of instruments and sampling methods to identify operational defects prior to main deployment.