Critical Thinking in Psychology: Surveys, Observations, and Sampling

Vocabulary flashcards covering survey question formats, question flaws, response sets, observational research, and probability versus nonprobability sampling techniques from Chapters 6 and 7.

Construct Validity (in Survey Research)

An assessment of whether survey questions are designed, worded, formatted, and ordered in a way that allows respondents to answer accurately and meaningfully, measuring what they are intended to measure.

External Validity (in Sampling)

An assessment of how well the findings from a sample generalize to the defined population of interest, governed by the sampling technique rather than sample size.

Open-Ended Question

A survey question format that allows respondents to answer in any way they choose, providing rich and detailed data but requiring researchers to code responses.

Forced-Choice Question

A question format where respondents must choose between specific preset options (such as "Yes" or "No"), making results easy to summarize while discarding intermediate opinions.

Likert Scale

A survey response scale where respondents indicate their level of agreement with a statement using anchors ranging from "strongly disagree" to "strongly agree."

Semantic Differential Format

A survey rating scale where participants evaluate an object, person, or concept along a numeric continuum anchored by opposite adjective pairs (e.g., boring to interesting).

Leading Question

A question whose phrasing subtly or overtly pushes the respondent toward a particular answer, systematically biasing construct validity.

Double-Barreled Question

A poorly constructed survey item that combines two separate questions into a single item, making it impossible to determine which issue the respondent's answer addresses.

Negatively Worded Question

A question that uses complicated negative phrasing or double negatives, creating cognitive confusion and reducing construct validity.

Question Order Effect

A phenomenon where an earlier question in a survey primes respondents or provides a context that changes how a subsequent question is interpreted and answered.

Counterbalancing

A methodological procedure used to detect and control for question order effects by randomizing or alternating the sequence of questions across respondents.

Response Set

A shortcut strategy where survey respondents answer items on autopilot using a uniform pattern rather than carefully reading and considering each question.

Acquiescence (Yea-Saying)

A response set in which a participant selects the affirmative or agreeing response for every survey item regardless of its content.

Fence Sitting

A response shortcut where a respondent repeatedly chooses the neutral or midpoint option for every question to avoid taking a stance.

Reverse-Keyed Items

Survey questions worded in the opposite conceptual direction of the construct so that agreement reflects the opposite attitude, used to detect or break acquiescence.

Socially Desirable Responding (Faking Good)

Giving responses designed to create a favorable impression rather than truthful answers, particularly prevalent on sensitive topics like prejudice or drug use.

Observational Research

A measurement approach where researchers systematically record actual behaviors rather than asking participants to self-report them.

Observer Bias

A bias that occurs when an observer's expectations influence how they see, interpret, or record participant behavior.

Observer Effects

A phenomenon where an observer's expectations unintentionally alter participant behavior so that the participant acts in a way that confirms those expectations.

Clever Hans

A horse reputed to perform arithmetic whose behavior was actually shaped by observer effects; he tapped his hoof until detecting subtle, unintentional physical cues from his questioner.

Masked Design (Blind Design)

An observational study design where observers are unaware of the study hypotheses or the specific experimental condition of the participants to prevent observer bias and observer effects.

Codebook

A precise set of explicit rules and definitions that specify how target behaviors are identified and categorized, ensuring high interrater reliability across observers.

Reactivity

A change in participant behavior that occurs simply because they are aware of being watched or evaluated.

Unobtrusive Observation

An observation technique where the researcher remains hidden (e.g., behind a one-way mirror) to prevent participants from knowing they are being observed.

Population of Interest

The entire group of individuals, events, or items that a researcher wants to generalize their findings and claims about.

Sample

A smaller subset of individuals drawn from the defined population of interest that is actually studied in research.

Census

A data collection method in which every single member of the population of interest is tested or observed.

Probability Sampling

Any sampling technique where every individual in the population of interest has a known, non-zero chance of being selected, enabling statistical generalization.

Simple Random Sampling

A probability sampling method where every member of the population has an equal chance of being selected through a completely random process.

Systematic Sampling

A probability sampling technique where a researcher selects a random starting case and then recruits every $n\text{th}$ individual from a list.

Cluster Sampling

A probability sampling technique where researchers randomly select entire pre-existing groups (clusters) and then include every individual within those selected clusters.

Multistage Sampling

A probability sampling technique that involves two random steps: first randomly selecting clusters, and then randomly sampling individuals from within the selected clusters.

Stratified Random Sampling

A probability sampling method where the population is divided into meaningful categories (strata) and individuals are randomly sampled within each stratum to mirror population proportions.

Oversampling

A stratified sampling variant where a researcher deliberately samples a small subpopulation at a higher rate than its population proportion to obtain a reliable estimate, later weighting it back for population figures.

Nonprobability Sampling

Any sampling technique in which individuals do not have a known chance of selection, leaving the generalizability of results unknown.

Convenience Sampling

A nonprobability sampling method in which the researcher selects individuals who are most easily accessible to participate.

Self-Selection

A sampling bias occurring when a sample consists solely of volunteers who choose to participate, often overrepresenting people with strong opinions.

Undercoverage

A sampling flaw in which certain segments of the population of interest are systematically excluded from the sampling frame.

Purposive Sampling

A nonprobability sampling approach where researchers deliberately recruit only participants belonging to a specific target group of interest.

Snowball Sampling

A nonprobability sampling method where existing participants are asked to identify and recruit other people they know who meet the study criteria.

Quota Sampling

A nonprobability sampling method where researchers set specific numeric targets for target categories and recruit whoever is available until each quota is met without random selection.

Random Sampling vs. Random Assignment

Random sampling governs how participants are selected from a population to support external validity; random assignment governs how participants are placed into experimental conditions to establish internal validity.

Margin of Error

A statistical calculation describing the precision of an estimate derived from a random sample, reflecting sample size rather than sample representativeness.