Psychological Research Methods and Experimental Design

Comprehensive vocabulary flashcards covering experimental methodologies, measurement reliability and validity, survey construction, factorial designs, and threats to internal validity.

Experiment

A research design that establishes causality by manipulating one or more independent variables and assigning participants to different conditions while holding non-manipulated variables constant.

Confound (Confounding Variable)

An extraneous variable that covaries with the predictor variable, making it impossible to determine whether the intended predictor or the confound produced the observed effect.

Internal Validity

The extent to which a researcher is confident that the independent variable is the sole cause of observed changes in the dependent variable, free from the influence of confounds.

Construct Validity

The extent to which the operationalization of a variable accurately and faithfully represents the theoretical definition of that construct.

External Validity

The extent to which findings and results can generalize across different populations, operationalizations, settings, and cultures.

Statistical Conclusion Validity

The accuracy of inferences made regarding whether observed effects reflect real relationships rather than chance alone, typically evaluated with a threshold of $p < 0.05$.

Three Criteria for Causality

The three requirements necessary to substantiate a causal claim: covariance, temporal precedence, and internal validity (ruling out alternative explanations or confounds).

Reliability

The consistency, reproducibility, or stability of a measure of behavior over repeated observations or testings.

Test-Retest Reliability

A method of measuring reliability by administering the same measure to the same participants at two or more separate points in time and correlating the resulting scores.

Internal Consistency

The degree to which multiple items in a single scale measuring the same construct correlate with one another at a single point in time, evaluated using Cronbach's alpha ($\alpha > 0.8$).

Interrater Reliability

The degree of consensus or agreement between independent observers or coders assessing identical behaviors or artifacts, measured using percent agreement or Cohen's kappa.

Face Validity

The degree to which the content of a measurement superficially and visibly appears to measure the underlying construct.

Content Validity

The extent to which the items or tasks in a measurement comprehensively cover the entire domain and full range of dimensions defining a construct.

Predictive Validity

The degree to which scores on a measure accurately anticipate or predict future behavior, criterion outcomes, or performance.

Concurrent Validity

The degree to which scores on a measure relate to a relevant criterion, known groups comparison, or outcome measured simultaneously.

Convergent Validity

The extent to which a measure demonstrates strong correlations with other measures assessing the same or theoretically related constructs.

Discriminant Validity

The extent to which scores on a measure show weak or non-existent correlations with measures of theoretically distinct, unrelated constructs.

Response Sets

A systematic response bias where participants answer all survey questions from a specific, habitual mindset or perspective, such as acquiescence or socially desirable responding.

Double-Barreled Question

A problematic survey question that asks two distinct questions simultaneously within a single prompt, preventing clear interpretation of the respondent's answer.

Ceiling Effect

A measurement artifact where task items are so easy or scale endpoints so constrained that almost all participants obtain the highest possible score, obscuring true group differences.

Floor Effect

A measurement artifact where task items are excessively difficult or insensitive, causing nearly all participants to obtain the lowest possible score.

Total Variance

The total observed variability in scores, mathematically comprised of systematic variance caused by the independent variable plus error variance caused by extraneous variables and noise.

Type I Error

A false positive error occurring when a researcher concludes that an effect or relationship exists when in reality there is no true effect.

Type II Error

A false negative error occurring when a researcher fails to detect a genuine effect or relationship that is actually present.

Between-Subjects Design

An experimental framework where each participant is exposed to only one level or condition of the independent variable, requiring distinct groups for comparison.

Within-Subjects Design

An experimental framework where every participant is exposed sequentially to all levels or conditions of the independent variable.

Carryover Effects

Adverse influences on participant performance caused by previous conditions in a repeated-measures design, most commonly manifested as fatigue or learning effects.

Counterbalancing

An experimental technique in within-subjects designs where the administration order of conditions is systematically varied across participants to eliminate order and carryover biases.

Matched Group Design

A design where participants are paired based on specific extraneous variables that affect the dependent variable, with each member of the pair randomly assigned to a separate condition.

Factorial Design

An experimental structure featuring two or more independent variables (factors), allowing simultaneous assessment of their individual main effects and joint interactions.

Main Effect

The isolated, overall effect of a single independent variable on the dependent variable, averaged across all levels of all other independent variables in a factorial design.

Interaction

A pattern in factorial research where the effect of one independent variable on the dependent variable changes depending on the specific level of another independent variable.

Placebo Effect

A phenomenon wherein participants show genuine changes in physiological or psychological outcomes resulting merely from expectations rather than active treatment components.

Double-Blind Experiment

An experimental protocol in which both the participants and the experimenters interacting with them remain unaware of who is receiving the treatment versus the control.

Pilot Testing

The preliminary trial of experimental protocols, manipulations, or survey instruments on a small group of participants to confirm comprehension and sensitivity before full execution.

Manipulation Check

An explicit secondary measure embedded within an experiment to directly confirm whether an independent variable manipulation successfully altered the intended psychological construct.

Suspicion Probe

A structured series of debriefing questions (often funneled from broad to narrow) used to assess whether participants guessed the true experimental hypothesis or purpose.

Cover Story

A plausible but false explanation provided to study participants regarding the research objectives in order to minimize demand characteristics and suspicion.

Confederate

An actor working secretly with the experimenter who plays the role of another research subject or bystander while executing pre-scripted behaviors.

Regression to the Mean

A statistical phenomenon in pretest/posttest studies whereby extreme initial performance naturally drifts back toward the average during subsequent measurements due to random error.

Attrition

The systematic or unsystematic dropping out of participants from an investigation prior to its completion, which threatens internal validity if attrition rates differ across groups.

Trait Measure

A psychological assessment designed to capture enduring, persistent behavioral and personality characteristics (such as extraversion or neuroticism).

State Measure

A psychological assessment designed to capture temporary, transient feelings or physiological conditions that fluctuate across situations (such as current mood or confidence).