Sampling and Generalizability pt 2

What is quota sampling?

Filling target numbers for categories (e.g., gender) using nonrandom recruitment; resembles stratified structure without randomness.

Does increasing sample size fix a biased sampling method?

No; a large biased sample remains unrepresentative.

What does sample size primarily affect?

Statistical validity (precision and margin of error), not external validity.

When do large samples matter most?

When estimating rare phenomena or small subgroups that need many observations.

What is margin of error, and how does sample size influence it?

It’s the confidence interval around an estimate; larger samples reduce it, with diminishing returns beyond ~1,000 people.

What is a census?

Data collection from every member of a population (usually impractical).

What is the 'population of interest'?

The specific group a study aims to generalize to (e.g., 'eligible voters in the next election').

Why isn’t 'coming from a population' the same as 'representing it'?

A sample can still over/underrepresent subgroups and fail to generalize accurately.

How have pollsters adapted to low phone response rates?

By forming randomly recruited online panels via mailed invitations and inclusive access.

When can biased Internet samples still be useful?

If the biasing trait is unrelated to the outcome (e.g., shoe-fit ratings, traffic app reports), the descriptive data can still generalize.

How do probability and nonprobability sampling differ in generalizability?

Probability methods support confident generalization; nonprobability methods leave external validity uncertain.

For which claim types does external validity matter most?

Frequency claims; it matters less for association and least for causal claims.

What key question should researchers ask about biased samples?

Whether the cause of bias is related to the outcome; if not, generalization may still be reasonable.

Give an example of stratified sampling in Canada.

Sampling South Asian Canadians in proportion to their national prevalence to preserve population ratios.

How is oversampling corrected in analysis?

By applying statistical weighting to restore true population proportions.

Why is quota sampling not equivalent to stratified sampling?

Quota uses nonrandom selection to fill categories; stratified uses random selection within strata.

In the bookstore exercise, what would a stratified sample look like?

Sampling books proportionally across subjects (e.g., psych, math, law) to reflect the store’s inventory.

In the bookstore exercise, what is a systematic sample?

Selecting every 60th book after a random start (e.g., start at the 10th).

Why can a small random sample outperform a large biased one?

Random selection captures the population structure; bias does not—regardless of size.