insh6300 - week 5 (Survey Methods)

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/99

flashcard set

Earn XP

Description and Tags

Comprehensive vocabulary flashcards covering sampling designs, probability vs non-probability sampling, questionnaire design, administration modes, data quality checks, nonresponse weighting, and total survey error.

100 Terms

1
New cards

Survey

A set of questions that is designed and administered to a group of people in a systematic and standardized way.

2
New cards

Systematic Data Collection

A feature of surveys where data collection adheres to formal, pre-determined procedures designed to ensure the collection of quality data.

3
New cards

Standardized Data Collection

A feature of surveys where every respondent receives the same questions, asked the same way, so that responses are comparable across individuals.

4
New cards

Target Population

The entire group of people or units that a researcher aims to learn about and draw conclusions on through a survey.

5
New cards

Sample

A selected subset of people or units drawn from a target population to participate in a survey.

6
New cards

Sampling

The operational process of selecting a sample from a target population.

7
New cards

Census

A study design in which the entire target population is surveyed, such that the sample is equal to the population (sample=population\text{sample} = \text{population}).

8
New cards

Non-probability Sampling

Any sampling technique in which the probability of any individual in the target population being selected into the sample is unknown.

9
New cards

Convenience Sampling

A non-probability sampling method where respondents are chosen based on ease of accessibility or proximity (e.g., asking anyone near you).

10
New cards

Purposive Sampling

A non-probability sampling method in which the researcher deliberately selects sample units based on specific characteristics or subjective judgment.

11
New cards

Quota Sampling

A non-probability sampling method in which researchers sample individuals until a predetermined number or proportion of specific subgroups is achieved.

12
New cards

Snowball Sampling

A non-probability sampling technique where existing survey participants recruit future participants from among their acquaintances.

13
New cards

Sample Selection Bias

A systematic bias that occurs when the outcome of interest is related to an individual's likelihood of being included in the sample.

14
New cards

Probability Sampling

A sampling technique in which every unit in the target population has a known, nonzero chance of being included in the sample through random selection.

15
New cards

Random Selection

A sample selection mechanism where inclusion into the sample is determined purely by random chance.

16
New cards

Nonzero Probability of Selection

The probability sampling condition requiring that every unit in the target population has a mathematical chance greater than zero (P>0P > 0) of being sampled.

17
New cards

Sampling Frame

The list or procedure used to identify and select units from the target population.

18
New cards

Ideal Sampling Frame

A list containing a single record for every individual in the target population (and no one outside it), containing contact information and unit attributes.

19
New cards

Simple Random Sampling (SRS)

A probability sampling method where sample units (e.g., individuals, businesses, addresses) are picked one at a time based purely on random chance, analogous to a lottery.

20
New cards

SRS Selection Probability

The uniform selection probability in simple random sampling, calculated as sample sizepopulation size\frac{\text{sample size}}{\text{population size}} (e.g., 10030,000=1300\frac{100}{30{,}000} = \frac{1}{300}).

21
New cards

Non-representative Sample by Random Chance

The risk in simple random sampling where random chance inadvertently yields an imbalanced sample (e.g., selecting too few freshmen).

22
New cards

Systematic Random Sampling

A sampling method involving picking a random number to start with and then selecting every kkth unit on a list, where k=population sizesample sizek = \frac{\text{population size}}{\text{sample size}}.

23
New cards

Sampling Interval (kk)

The fixed selection step size in systematic sampling, calculated by dividing the population size by the sample size (e.g., 30,000100=300\frac{30{,}000}{100} = 300).

24
New cards

Random Start (Systematic Sampling)

The randomly selected initial number within the first interval (11 to kk) designating the first unit included in a systematic sample (e.g., row 179179).

25
New cards

Sorted Sampling Frame (Systematic Sampling)

Arranging a sampling frame by a relevant variable (e.g., class year) prior to systematic sampling to ensure every group is represented in proportion to its size.

26
New cards

Periodicity

A vulnerability in systematic sampling where a repeating cyclical pattern in the list matches or divides evenly into interval kk, skewing the sample (e.g., every 10th house is a corner house).

27
New cards

List Order Dependency

A drawback of systematic sampling where results depend on list order, making it impossible to know if the arrangement helps or hurts if the ordering is unknown.

28
New cards

Stratified Random Sampling

A probability sampling method where the population is divided into subgroups (strata) by known characteristics (e.g., age, sex, grade level), followed by independent random sampling within each stratum.

29
New cards

Stratum

A homogenous subpopulation defined by shared known characteristics from which sample units are randomly selected in stratified sampling.

30
New cards

Proportionate Stratified Sampling

A stratified sampling method where units are drawn from each stratum at the exact same rate (e.g., 100/30,000=1/300≈0.0033333100/30{,}000 = 1/300 \approx 0.0033333), preserving population proportions.

31
New cards

Disproportionate Stratified Sampling

A stratified sampling design where different strata have unequal selection probabilities, producing subgroup sample proportions that differ from the target population.

32
New cards

Oversampling Subgroups

A form of disproportionate stratified sampling where one or more groups have a selection probability greater than their representation in the target population.

33
New cards

Selection Weight

A statistical weight equal to the inverse of a respondent's selection probability (1÷probability1 \div \text{probability}) used to restore groups to their true population share.

34
New cards

Multistage Cluster Sampling

A probability sampling design that involves selecting separate samples from multiple groups or clusters of units across successive hierarchical stages.

35
New cards

Cluster (Sampling Unit)

A naturally occurring grouping or collective unit of the population (e.g., high schools, city blocks, or counties) sampled in cluster sampling.

36
New cards

Cluster Heterogeneity vs. Stratum Homogeneity

The structural distinction where strata in stratified sampling are designed to be homogenous, while clusters in cluster sampling should ideally be heterogeneous.

37
New cards

Loss of Precision in Cluster Sampling

The increase in standard errors and reduction in statistical precision that occurs with cluster sampling compared to simple or stratified random sampling.

38
New cards

National Crime Victimization Survey (NCVS)

A nationally representative survey measuring criminal victimizations experienced in the past 66 months using a stratified, multi-stage cluster design.

39
New cards

NCVS Target Population

People age 1212 and older living in U.S. households.

40
New cards

NCVS Reference Period

The past 66 months prior to the interview, during which victimizations reported by respondents occurred.

41
New cards

NCVS Sampling Unit

The physical residential address, rather than specific families or persons.

42
New cards

NCVS Multi-Stage Cluster Stages

The clustering hierarchy of the NCVS: Stage 1 = counties/metro areas; Stage 2 = areas of 750–1,500750\text{--}1{,}500 people; Stage 3 = clusters of housing units.

43
New cards

NCVS Household Interview Protocol

The interview rule in the NCVS requiring that all household members aged 1212 and older residing at the selected address are interviewed.

44
New cards

Panel of Addresses

A longitudinal survey structure (used in the NCVS) where physical housing units remain in the sample over time, rather than tracking specific persons or families.

45
New cards

Questionnaire Design Process

A three-step development sequence: 1) choosing a preliminary set of questions, 2) evaluating the questions, and 3) pretesting the questionnaire in the field.

46
New cards

Focus Group Discussions (Questionnaire Design)

Facilitated discussions conducted with target population members during preliminary question development to ensure questions are simple to understand and answer.

47
New cards

Cognitive Interviews

An evaluation technique where respondents think aloud while answering draft questions to help researchers detect flaws and comprehension difficulties.

48
New cards

Field Pretesting

Administering a draft survey under field conditions to assess interviewer delivery, respondent clarification requests, and answer adequacy.

49
New cards

Social Desirability Bias

The tendency of respondents to answer survey questions inaccurately in order to look good in front of the person administering the survey.

50
New cards

Self-Administered Survey

A survey format completed directly by the participant without an interviewer present, recommended for sensitive questions to minimize social desirability.

51
New cards

Aural vs. Visual Survey Mode Design

The design requirement that questions must match the mode: simpler questions for aural modes (listening), and layout design features for visual modes (reading).

52
New cards

Web Survey Mode (Strengths)

Low cost per complete, fast deployment, capability for complex skip logic and visuals, and privacy for sensitive questions.

53
New cards

Web Survey Mode (Limitations)

Lack of a general email sampling frame, lower response rates, exclusion of offline populations, and vulnerability to bots and fraud.

54
New cards

Mail Survey Mode (Strengths)

Address lists cover nearly all households, offers privacy, and allows respondents to set their own completion pace.

55
New cards

Mail Survey Mode (Limitations)

Slow turnaround times, printing and postage costs, limited skip logic capacity, and inability to confirm who completed the questionnaire.

56
New cards

Phone Survey Mode (Strengths)

Interviewers can clarify questions and probe answers, and surveys can be fielded quickly.

57
New cards

Phone Survey Mode (Limitations)

Very low response rates that drive up costs, necessity for short and simple questions, and increased vulnerability to social desirability bias.

58
New cards

In-Person Survey Mode (Strengths)

Highest response rates, capacity to handle long and complex interviews, and option for respondents to enter sensitive answers privately on a laptop.

59
New cards

In-Person Survey Mode (Limitations)

The most expensive and slowest survey administration mode, along with vulnerability to interviewer effects.

60
New cards

Interviewer Effects

Distortions introduced into survey responses caused by the presence, demeanor, tone, or characteristics of an interviewer.

61
New cards

Mixed-Mode Survey Design

The practice of using multiple contact modes, response modes, or both to leverage the strengths of certain modes and overcome weaknesses of others.

62
New cards

Contact Mode vs. Response Mode

The distinction between how a respondent is reached (e.g., an email invitation) versus the medium used to complete the survey (e.g., online web form).

63
New cards

Measurement Problem (Mixed-Mode Surveys)

The methodological complication where the exact same question produces different answers across different modes, confounding mode effects with true values.

64
New cards

McMaster et al. (2017)

A study evaluating survey response performance by comparing web-push surveys against paper-only surveys.

65
New cards

Unit Nonresponse

The failure to obtain any survey participation from an individual or unit selected into the sample.

66
New cards

Item Nonresponse

When an active survey participant fails or refuses to answer a specific question within the questionnaire.

67
New cards

Nonresponse Rate

The percentage of sampled units that did not participate in or complete the survey.

68
New cards

Gaining Access to Sampled Individuals

Methods to reach sampled units by making repeated contact attempts during different days and times, and employing interviewers with flexible schedules.

69
New cards

Getting Sampled Individuals to Agree

Methods to encourage participation, including advance letters, small cash incentives, selling survey value, limiting burden, and professional presentation.

70
New cards

Advance Letter

An informative letter sent prior to survey administration to establish credibility, explain the survey's value, and improve agreement to participate.

71
New cards

Cash Incentive (Advance)

A small financial token provided upfront with the invitation, recognized as particularly effective for boosting response rates.

72
New cards

Cognitive Burden

The mental effort and time required by a respondent to interpret, process, and complete a survey, which should be minimized.

73
New cards

Imputation

A statistical procedure used to estimate and fill in missing values resulting from item nonresponse.

74
New cards

Nix et al. (2019)

An empirical study focusing on police research, officer surveys, and methodologies for improving response rates.

75
New cards

Data Quality Check (DQC)

A procedure used to detect, prevent, or correct errors, inattentive responses, or invalid submissions in survey datasets.

76
New cards

Real-Time DQC

Quality control procedures built directly into digital surveys that prompt respondents to rectify errors or inconsistencies before submission.

77
New cards

CAPTCHA Test

A challenge-response security verification used in web surveys to block automated software and bots from submitting responses.

78
New cards

Single-Use Survey Link

An individualized survey URL that restricts completion to one submission, preventing repeat responses and preventing links from being shared.

79
New cards

Survey Farmers

Human respondents who complete surveys for which they are not eligible solely to collect financial incentives.

80
New cards

Westwood (2025)

Research showing that AI agents can bypass most automated quality checks, concluding that sample design (who gets invited) is the strongest defense.

81
New cards

Range Check

A data quality check that flags numerical responses falling outside possible or plausible bounds (e.g., reporting an age of 500500).

82
New cards

Consistency Check

A data quality check comparing an answer to an earlier response to detect logical contradictions (e.g., prisoners by race not summing to total prisoners).

83
New cards

Straightlining Flag

A data quality check identifying inattentive respondents who select the same response across a set of items, including reverse-worded questions.

84
New cards

Speeding Flag

A data quality check that flags respondents who complete pages too quickly (e.g., completing a 5-minute page in <1< 1 minute).

85
New cards

Disengagement Flag

A quality indicator identifying respondents who appear uninvested based on high numbers of skipped items or 'don't know' responses.

86
New cards

Honeypot Question

A hidden survey question invisible to human respondents but filled out by automated bots, used for bot identification.

87
New cards

Duplicate Indicator

Tracking metrics such as recorded IP addresses and device fingerprints used to identify multiple submissions from the same respondent.

88
New cards

Douglas et al. (2023)

A scholarly study examining data quality maintenance and threats in online human-subjects research.

89
New cards

Differential Nonresponse

The condition where specific demographic or behavioral subgroups in a sample fail to participate to a greater extent than other subgroups.

90
New cards

Poststratification

A weighting technique used to adjust a post-survey sample to make it match known population characteristics on selected variables.

91
New cards

Poststratification Assumption

The assumption that respondents from a given subgroup are systematically similar to nonrespondents from that same subgroup.

92
New cards

Poststratification Weight Calculation Example

In a 50%50\% male / 50%50\% female population where a sample of 400400 is 40%40\% male (N=160N = 160) and 60%60\% female (N=240N = 240), males are weighted by 0.5/0.4=1.250.5 / 0.4 = 1.25 (160×1.25=200160 \times 1.25 = 200) and females by 0.5/0.6≈0.8330.5 / 0.6 \approx 0.833 (240×0.833≈200240 \times 0.833 \approx 200).

93
New cards

Total Survey Error (TSE)

The conceptual framework stating that the total difference between a survey estimate and the true population value is the sum of representation errors and measurement errors.

94
New cards

Coverage Error

An error of representation that occurs when the sampling frame misses eligible population members or includes people outside the target population.

95
New cards

Sampling Error

An error of representation reflecting chance variation that arises from surveying a sample rather than the entire population.

96
New cards

Nonresponse Error

An error of representation that occurs when participating survey respondents differ systematically from nonrespondents.

97
New cards

Adjustment Error

An error of representation that occurs when post-survey adjustments, such as weighting, fail to fully correct differences between sample and population.

98
New cards

Validity (Survey Measurement)

An error of measurement that occurs when a survey question fails to accurately capture or reflect the intended theoretical concept.

99
New cards

Measurement Error

An error of measurement caused by question wording, administration mode, interviewer characteristics, social desirability bias, or recall error.

100
New cards

Processing Error

An error of measurement resulting from mistakes in data entry, coding, editing, programming, or data cleaning.