Sampling and Research Design

Population

The entire set of elements to which study findings are to be generalized

Sampling Frame

A list of all the elements in a population

Representative Sample

A sample that reflects the characteristics of the population accurately

Probability Sampling

Sample selected using random selection. Researchers know the likelihood of an element being selected. Minimizes systematic bias

Random Sampling

Every element is selected based solely on chance, having an equal probability of selection

Systematic Sampling

Choosing every nth unit from a random starting point

Stratified Sampling

Elements are selected proportionally from strata (subgroups) to ensure representation in the sample

Cluster Sampling

Multistage sampling: First, randomly select naturally occurring clusters (e.g., schools/cities), then randomly select elements within clusters. Useful when a sampling frame is unavailable

Non-Probability Sampling

Sample selected without random selection. Representativeness cannot be determined

Types of Non-Probability Sampling

• Convenience/Availability: Selection based on ease; rarely representative (e.g., UvA students, paid panels like Prolific).

• Quota: Selecting elements to ensure proportions of certain characteristics are met (non-random final selection).

• Purposive: Selecting elements for a specific purpose/unique characteristics (e.g., key informants).

• Snowball: Elements identified by initial informants/interviewees; useful for hard-to-reach groups

Cross-Sectional Design

Data collected at a single point in time. Examines relationships (correlation), but provides no conclusions about causality (weak internal validity)

Longitudinal Design

Data collected at two or more points in time. Types: Panel study (same individuals followed over time); Cohort study/Event-based (different samples from a group sharing a common starting point, e.g., birth year)

Causality Criteria

1. Association (variation in IV is related to DV).

2. Time Order (IV precedes DV).

3. Nonspuriousness (relationship is not caused by an extraneous/third variable)

True Experimental Design

Gold standard for testing causal effects. Essential components: Comparison groups (treatment/control), Measurement of change (pre/post-test or Time 2), and Random Assignment

Random Assignment (Randomization)

Procedure where each subject has an equal probability of being assigned to any condition. Crucial for Internal Validity; neutralizes selection bias by equating groups on individual factors

Random Sampling vs. Random Assignment

Random Sampling (Selection): Ensures Sample Generalizability (external validity to the population).

Random Assignment (Group Placement): Ensures Internal Validity (causality)

Experimental Designs

Between Subjects: Participants assigned to conditions; each receives a different manipulation.

Within Subjects: Every participant gets all different manipulations (serves as own control)

Manipulation Check

Measures whether the independent variable successfully manipulated the theoretical concept. Can potentially introduce bias (demand characteristics)

Threats to Internal Validity (Causality)

Selection Bias: Groups differing due to non-random assignment.

Confounds: Variables other than IV/DV influencing findings.

Maturation/Testing/History/Mortality: Natural changes, practice effects, external events, or selective dropout.

Experimenter Bias: Experimenter's actions influence responses.

Demand Characteristics: Participants adjust behavior based on hypothesized expectations

Quasi-Experiments

Comparison group is similar to experimental group, but subjects are not randomly assigned. Used when true random assignment is impossible or unethical