Sampling Methods and Generalizability (Random Sampling vs. Random Assignment)

Random Sampling vs Random Assignment

• Random Sampling: How you choose people from the population to be in your study.

  • Affects generalizability.

• Random Assignment: How you place participants into experimental vs. control groups after they're already in the study.

  • Affects internal validity and allows cause-and-effect conclusions.

• Examples:

  • Random sampling: Picking 100 students randomly from a school of 1,000.
  • Random assignment: Flipping a coin to decide who gets caffeine vs. placebo.

• Bias in Sampling:

  • Sampling Bias: When the sample doesn't represent the population.
  • Volunteer Bias: Only motivated people join.
  • Convenience Bias: Easy access group may not represent the population.
  • Example: If only athletes volunteer for a study on stress, results can't be generalized to all students.

• Key Exam Tips:

  • If the sample is random → results are more generalizable.
  • If the sample is from only one group (school, clinic, city) → generalize ONLY to that group.
  • Important distinction: Random assignment is not the same as random sampling. Don’t mix them.
  • The bigger the sample, the more likely it reflects the population.
  • Case studies = in-depth but NOT generalizable.
  • Correlational studies = show relationships but NOT cause and effect.

• Generalizability:

  • Definition: The extent to which findings from a study can be applied to the larger population.
  • High generalizability comes from random, representative samples.
  • Low generalizability comes from biased, small, or convenience samples.
  • Example: Studying sleep habits of 1,000 randomly chosen U.S. teens → high generalizability.
  • Example: Studying sleep habits of 30 honors students in one private school → low generalizability.

Sampling Methods

• 1. Random Sampling

  • Definition: Every member of the population has an equal chance of being chosen.
  • Strength: Most representative → highest generalizability.
  • Example: Assign numbers to all students in a school and use a random number generator to pick 100.
  • Math note: In simple random sampling, each individual has probability p = \frac{n}{N} where n is the sample size and N is the population size.

• 2. Stratified Sampling

  • Definition: The population is divided into subgroups (strata) based on characteristics (e.g., grade, gender).
  • Participants are randomly selected proportionally from each group.
  • Strength: Ensures important subgroups are represented.
  • Example: If a school is 60% female and 40% male, the researcher ensures the sample reflects that ratio.

• 3. Systematic Sampling

  • Definition: Select every nth person from a list or roster.
  • Strength: Easy and quick.
  • Limitation: Can create bias if there is a hidden pattern in the list/order.
  • Example: Every 10th name in the school yearbook is selected.

• 4. Convenience Sampling

  • Definition: Using whoever is easiest to access.
  • Strength: Quick and cheap.
  • Limitation: Usually biased, low generalizability.
  • Example: A researcher surveys only her psychology class.

• 5. Volunteer (Self-Selected) Sampling

  • Definition: Participants choose themselves by responding to an ad, flyer, or request.
  • Strength: Easy to gather participants.
  • Limitation: Biased sample — only motivated people volunteer.
  • Example: Posting a flyer in the library asking for caffeine study volunteers.

• 6. Cluster Sampling (less common, sometimes tested)

  • Definition: Divide population into clusters (e.g., classrooms, neighborhoods) and randomly choose whole clusters.
  • Strength: Efficient when population is spread out.
  • Limitation: May not represent diversity within each cluster.
  • Example: Randomly selecting 5 classrooms in a school and surveying every student in those rooms.

Generalizability and Sampling Methods (Detailed)

• Generalizability (revisited): The extent findings can be applied to the larger population.
• Random Sampling → highest generalizability (if well-implemented).
• Stratified Sampling → helps ensure representation of key subgroups, increasing generalizability across those subgroups.
• Systematic Sampling → convenient but risks bias if order has a pattern.
• Convenience, Volunteer, and Cluster Sampling → generally lower generalizability due to biases and potential lack of diversity within samples.

Quick Reference: Correlation vs. Causation (contextual reminder)

• Correlational studies: Show relationships between variables but do NOT establish causation.
• Experimental studies with random assignment: Can support cause-and-effect conclusions by controlling for confounding factors.

Practical Implications and Ethical Considerations

• Choosing a sampling method should balance:
  • Representativeness and generalizability
  • Practical constraints (time, cost, access)
  • Ethical considerations in recruiting and inclusion criteria
• Be transparent about limitations: sample bias, lack of diversity, and generalizability boundaries.

Formulas and Notation Summary

• Probability of selection in simple random sampling:
  • p = \frac{n}{N}
  • where n = sample size, N = population size.
• When using stratified sampling, allocation preserves population proportions across strata for representative sampling.