Research Methods: Sampling, Experimental Designs, and Multivariate Correlations

Comprehensive vocabulary flashcards covering sampling techniques, experimental designs, threats to internal validity, and multivariate correlational models (longitudinal designs, multiple regression, moderation, and mediation).

Population of Interest

The entire group of individuals or cases to which a research theory, hypothesis, or conclusion is intended to apply.

Sample

The subset of individuals selected from the population of interest who are actually observed or participate in a study.

External Validity

An indicator of how well the results of a study generalize to other people, settings, contexts, real-life situations, or studies conducted by other researchers.

Random Sampling

A probability sampling approach where every member of the population of interest has a known and generally equal chance of being selected, which supports high representativeness.

Simple Random Sampling

A basic probability sampling method where the researcher identifies the entire population of interest and randomly selects participants, such as by using a random-number generator.

Systematic Sampling

A probability sampling technique where a researcher calculates a sampling interval $k$ by dividing the population size by the desired sample size, then selects every $k\text{th}$ individual from a list.

Cluster Sampling

A probability sampling method in which naturally occurring groups (clusters) within a population are identified, a subset of those clusters is randomly selected, and every individual in the selected clusters is recruited.

Multistage Sampling

A probability sampling method that begins by randomly selecting clusters from a population, followed by an additional stage of randomly sampling individuals from within each selected cluster.

Stratified Random Sampling

A probability sampling method where the researcher identifies demographic subgroups (strata) and randomly samples individuals within each subgroup so that the sample's proportions match those in the population.

Oversampling

A probability sampling technique where a researcher intentionally recruits a larger proportion of a rare or underrepresented subgroup than exists in the population to ensure sufficient sample size for analysis.

Convenience Sampling

A non-probability sampling method that recruits individuals who are readily and practically accessible, such as through campus flyers, participant pools (e.g., SONA), or online crowdsourcing platforms.

Purposive Sampling

A non-probability convenience sampling technique that specifically targets a particular group of people relevant to the study's research question (e.g., recruiting from an addiction treatment center).

Quota Sampling

A non-probability sampling technique where a researcher identifies target subgroup proportions and uses convenience sampling until the preset quota for each category is filled.

Snowball Sampling

A non-probability sampling technique where existing study participants are asked to recruit additional participants from their own social networks, such as friends, family members, or classmates.

Self-Selection Bias

A sampling bias that occurs when individuals choose whether to opt into a study, resulting in a sample that differs systematically in relevant characteristics from individuals who decline to participate.

Non-Response Problem

A sampling challenge that occurs when individuals selected for a study fail to respond or complete it, potentially introducing bias if responders differ systematically from non-responders.

Statistical Power

The ability of a study to detect an effect or relationship between variables when one truly exists, which generally increases as sample size increases.

Independent Variable (IV)

The variable that is manipulated by the researcher in an experiment to test its causal influence on an outcome.

Dependent Variable (DV)

The outcome variable measured by the researcher to determine whether it changes as a function of the manipulated independent variable.

Experimental Control

The practice of holding all variables constant across conditions except for the manipulated independent variable.

Random Assignment

The experimental procedure of allocating participants to conditions purely by chance (e.g., coin flip or random-number generator) to distribute individual differences equally across groups.

Covariance

The first criterion for establishing a causal claim, demonstrating that the independent variable and dependent variable are statistically associated or related.

Temporal Precedence

The second criterion for establishing a causal claim, requiring that the cause (independent variable) occurs or is manipulated before the outcome (dependent variable) is measured.

Internal Validity

The third criterion for establishing a causal claim, reflecting the degree to which a study design successfully rules out plausible alternative explanations and confounds.

Confound

An extraneous variable that is related to the dependent variable and varies systematically with the levels of the independent variable, undermining internal validity.

Noise Variable

An extraneous variable or outlier that may relate to the dependent variable but varies unsystematically across conditions, reducing statistical power without threatening internal validity.

Selection Effect

A threat to internal validity in independent-groups designs where participants choose their condition or are assigned non-randomly, creating systematic group differences on participant characteristics.

Matched-Groups Design

An experimental assignment method where participants are measured on a relevant variable, paired with a similar participant, and randomly split between conditions to equalize the groups on that variable.

Independent-Groups Design

An experimental design (also termed between-groups) in which each participant is exposed to only one condition or level of the independent variable.

Within-Groups Design

An experimental design in which every participant is exposed to all levels or conditions of the independent variable, allowing each participant to serve as their own control.

Repeated-Measures Within-Groups Design

A within-groups design where participants complete one condition, have the dependent variable measured, and subsequently complete the other condition followed by another dependent variable measurement.

Concurrent-Measures Within-Groups Design

A within-groups design in which participants are exposed to all levels of the independent variable at the same time, followed by a single measurement of preference or behavior.

Order Effects

A threat to internal validity in within-groups designs where the sequence of conditions influences participant responses, encompassing carryover effects, fatigue effects, and practice effects.

Counterbalancing

A control procedure in within-groups designs where the presentation order of experimental conditions is randomized or varied across participants to neutralize order effects.

Manipulation Check

An extra dependent measure administered after a manipulation to verify whether the experimental manipulation effectively altered the intended psychological construct.

Demand Characteristics

Cues or features of an experiment that allow participants to deduce the study's hypotheses, leading them to alter their behavior to fit perceived expectations.

Placebo Effect

A phenomenon in which participants show genuine improvement simply because they believe they are receiving a valid, active treatment.

Maturation Threat

An internal validity threat in single-group pretest–posttest designs where observed changes in the dependent variable occur simply due to natural biological or psychological development over time.

History Threat

An internal validity threat in pretest–posttest studies where an external event occurring between test points influences participants' responses, confounding the effect of the manipulation.

Regression to the Mean

A statistical phenomenon whereby participants who score extremely high or extremely low at an initial measurement point tend naturally to score closer to the average on retesting.

Attrition Threat

An internal validity threat occurring when participants drop out of a study, especially problematic when those dropping out have extreme pretest scores or differ systematically from those who remain.

Testing Threat

An internal validity threat where taking a pretest influences participant performance on a subsequent post-test because participants remember answers or become sensitized to the measure.

Instrumentation Threat

An internal validity threat that arises when the measurement tool, observational criteria, or coders change or drift from pretest to posttest, creating an apparent difference in scores.

Longitudinal Design

A multivariate correlational design that measures the same variables across multiple time points in the same individuals to investigate temporal precedence.

Cross-Sectional Correlation

A correlation in a longitudinal design that evaluates the association between two distinct variables measured at the exact same point in time.

Autocorrelation

A correlation in a longitudinal design that measures the association of a variable with itself across two different time points, reflecting the variable's stability over time.

Cross-Lagged Correlation

A correlation in a longitudinal study that examines the relationship between an earlier measurement of one variable and a later measurement of another variable to help assess temporal precedence.

Bidirectional Relationship

A mutually reinforcing association in a longitudinal design where both cross-lagged correlations between two variables over time are statistically significant ($p < 0.05$).

Multiple Regression

A statistical analysis that evaluates the relationship between a predictor variable and an outcome while statistically controlling for one or more measured third variables (covariates).

Beta (\beta)

A standardized statistical coefficient in multiple regression that describes the strength and direction of the relationship between a predictor and an outcome while holding all other predictors constant.

Moderation

A multivariate correlational phenomenon where the strength or direction of the relationship between a predictor and an outcome variable depends on the level of a third variable (the moderator).

Mediation

A multivariate correlational model that tests the underlying theoretical mechanism or causal chain explaining why a predictor variable is related to an outcome variable.

Total Effect (Path C)

In a mediation analysis, the overall bivariate relationship between the predictor and the outcome before accounting for the mediator.

Indirect Effect (Path AB)

In a mediation analysis, the pathway through which the predictor influences the outcome via the mediator, conceptually calculated as Path $A \times \text{Path } B$.

Direct Effect (Path C')

In a mediation analysis, the remaining relationship between the predictor and the outcome after statistically controlling for the mediator ($C' = C - AB$).

Proportion Mediated

The fraction of the total effect explained by the mediator in a mediation model, calculated as the indirect effect divided by the total effect ($\frac{AB}{C}$).

Complete Mediation

A mediation outcome where the proportion mediated meets or exceeds the course threshold of $80\%$ ($0.80$), indicating that the mediator accounts for almost all of the predictor–outcome relationship.

Partial Mediation

A mediation outcome where the proportion mediated is below $80\%$ ($0.80$), indicating the mediator explains part of the relationship but other mechanisms likely exist.

Shared Method Variance

A factor affecting correlations where assessing both the predictor and outcome with the same measurement format (e.g., self-report questionnaires) artificially inflates their correlation.