Psychological Research Methods: Experimental Design, Validity, and Measurement

Essential vocabulary flashcards covering research methods in psychology, including epistemology, scientific design logic, between- vs. within-subjects designs, threats to internal and external validity, scales of measurement, reliability, validity types, and measurement artifacts.

Epistemology

The philosophical study of where knowledge comes from.

Rationalism

A method of knowing that uses formal logic (if A is true, then B is true) to draw conclusions from premises.

Empiricism

A method of knowing based on observation through the senses, either directly or indirectly.

Availability Heuristic

A cognitive bias where things that come easily to mind seem more likely because they are easily memorable.

Hindsight Bias

A cognitive bias where an outcome seems obvious only after it has already occurred and is known.

Confirmation Bias

The tendency to look for evidence that confirms preexisting beliefs while giving greater weight to consistent information.

Bias About Bias

The belief that other people possess cognitive biases while viewing oneself as objective and unbiased.

Working Memory

A memory store that holds all information currently in active use, integrating with long-term memory but possessing a strictly limited capacity.

Long-Term Memory

A memory repository for stored knowledge that is not constantly in active use, with no currently known capacity limit.

Generative Strategy Hypothesis

The hypothesis predicting that learning strategies requiring students to select, organize, and integrate material yield better learning outcomes than passive strategies.

Generative Processing

Active construction of knowledge involving elaboration, integration with prior knowledge, organizing concepts into comprehensible relationships, and generating connections.

Falsifiability

A defining characteristic of scientific theories requiring that a theory must generate testable hypotheses that could fail to be supported when evaluated with empirical data.

Parsimony

A scientific principle denoting a preference for the simplest explanation that accounts for observations.

Statistical Validity

The extent to which the statistical conclusions drawn from a study are accurate and reasonable.

Quantitative Research

A structured method of collecting and analyzing numerical data to test hypotheses, identify patterns, and generalize results to a larger population.

Qualitative Research

A method of scientific inquiry that collects and analyzes non-numerical data (words, audio, video, or text) to understand human behavior, experiences, opinions, and social contexts.

Correlational Research

A relational research strategy where variables are measured as they exist naturally without experimental manipulation or random assignment of participants.

True Experiment

An experimental research strategy in which the independent variable is actively manipulated and participants are randomly assigned to conditions, enabling causal claims.

Quasi-Experimental Research

A research approach in which participants are grouped into conditions based on preexisting characteristics rather than being randomly assigned.

Temporal Precedence

A required condition for establishing causality stating that changes in the independent variable must occur before the dependent variable is measured.

Covariation of Cause and Effect

A required condition for establishing causality stating that changes in the independent variable and dependent variable must co-occur.

External Validity

The degree to which the results of a study can be generalized beyond the sample and environment to other populations, studies, and real-world settings.

Sampling Bias

A threat to external validity occurring when non-representative samples make it difficult to generalize conclusions to the broader population.

Between-Subjects Design

An experimental design in which each participant completes only one condition, meaning treatment comparisons are made across distinct individuals.

Posttest-Only Design

A between-subjects design where participants are randomly assigned to a treatment condition and measured on the dependent variable once, following the intervention.

Pretest/Posttest Design

An experimental design in which participants are tested on the dependent variable twice: once before exposure to the independent variable and once after.

Assignment Bias

A confound occurring in between-subjects designs when the assignment process systematically produces groups with notably different participant characteristics.

Random Assignment

A control procedure where every participant has an equal chance of being placed in any condition independently of others, ensuring differences occur by chance alone.

Matching Groups

A restricted assignment method ensuring that experimental groups are made equivalent along specific participant characteristics that present high risk as confounds.

Holding Characteristics Constant

A control technique that prevents a potential confounding variable from varying at all across subjects (e.g., selecting only young adults), which boosts internal validity but reduces external validity.

Sampling Bias vs. Assignment Bias

Sampling bias stems from recruiting a sample non-representatively from the population, whereas assignment bias stems from systematically distributing participants into experimental conditions.

Differential Attrition

A between-subjects confound where subject drop-out rates differ systematically across conditions (e.g., treatment vs. control), distorting group comparisons.

Within-Subjects Design

An experimental design in which each subject participates in all experimental conditions, allowing each individual to serve as their own control.

Order Effects

A disadvantage of within-subjects designs where a participant's experience in a prior treatment condition alters their performance in subsequent conditions.

Progressive Error

An order effect caused by general progression through an experiment, leading to improved performance via practice or declining performance via fatigue.

Carryover Effects

A specific order effect where the lingering physiological or cognitive consequences of one condition directly affect behavior in a subsequent condition.

Counterbalancing

A method used in within-subjects designs to control order effects by altering the ordering of conditions across subjects so that order effects balance out on average.

Full Counterbalancing

A counterbalancing approach in which every possible condition ordering ($k!$, where $k$ is the number of conditions) is utilized in the study.

Partial Counterbalancing

A counterbalancing approach where each experimental condition appears exactly once in each sequential position across all orderings without requiring all $k!$ permutations.

Latin Square

A $k \times k$ matrix used in partial counterbalancing where each condition appears exactly once in each row and column, balancing progressive error and carryover effects.

Matched-Pairs Design

A design where participants are paired on key characteristics, with one member assigned to each group; technically between-subjects, but analyzed as within-subjects.

Maturation

A time-related confound occurring when subjects naturally change physiologically or psychologically over the time elapsed between measurements.

History

A time-related confound occurring when an outside environmental event takes place between treatment conditions and affects the dependent variable in subsequent measurements.

Regression to the Mean

A statistical phenomenon where extreme performance scores at Time 1 tend to naturally drift closer to the mean at Time 2 due to the non-repeatability of extreme combinations of factors.

Instrumentation

A time-related confound occurring when changes in measuring devices, calibration shifts, or alterations in observers between treatment conditions influence the dependent variable.

Construct

An internal psychological attribute or theoretical characteristic that cannot be observed directly (e.g., aggression, impulsivity).

Variable

A measurable quantity or quality that varies across individuals and can be directly observed.

Conceptual Definition

A theoretical description of a construct specified in terms of the underlying behaviors and internal psychological processes involved.

Operational Definition

A precise specification of how a variable will be measured and scored so that the observation can be used to infer the underlying construct.

Behavioral Measure

A measurement type in which the researcher directly observes and scores an aspect of participant behavior.

Self-Report Measure

A measurement type in which participants report and score their own subjective thoughts, feelings, or behaviors.

Physiological Measure

A measurement type that records a biological or physiological state of a participant's body, such as blood pressure or heart rate.

Nominal Scale

A scale of measurement consisting of categorical labels that have distinct names but no intrinsic quantitative ordering.

Ordinal Scale

A scale of measurement consisting of ordered sequential categories where the mathematical distance between adjacent values is not necessarily equal.

Interval Scale

A scale of measurement with equal intervals between adjacent values throughout the scale, but lacking a meaningful absolute zero point.

Ratio Scale

A scale of measurement with equal intervals between values and a meaningful absolute zero point representing the complete absence of the construct.

Measurement Reliability

The degree to which repeated measurements of the same entity yield consistent values across time, observers, and items.

Classical Measurement Theory

The theoretical model stating that any observed score is composed of the true underlying value plus measurement error ($\text{Measured Value} = \text{True Value} + \text{Error}$).

Inter-Rater Reliability

The extent to which two or more independent observers or coders agree when scoring the same set of observations.

Test-Retest Reliability

The consistency and correlation between scores obtained when the same participants complete the same measure across multiple points in time.

Internal Consistency

The degree of inter-item correlation between responses to different questions designed to assess the same underlying construct.

Measurement Validity

The degree to which an operational measurement process accurately captures the specific variable it is intended to evaluate.

Face Validity

A subjective evaluation of whether a measurement procedure superficially appears to capture the intended construct.

Content Validity

A subjective evaluation of whether a measurement instrument comprehensively covers all facets of a construct's conceptual definition.

Criterion Validity

An empirical form of validity assessed by testing for a statistical relationship between a measurement and an external behavioral criterion.

Convergent Validity

A form of criterion validity demonstrated when two different operational measures of the same underlying construct produce strongly correlated scores.

Divergent Validity

A form of criterion validity demonstrated when scores on a measure show little to no correlation with measures of conceptually distinct, unrelated constructs.

Predictive Validity

A form of criterion validity demonstrated when a measurement successfully predicts an individual's future behavior or outcomes.

Measurement Artifact

A non-natural feature or distortion introduced into an observation that would not occur outside the context of the study.

Experimenter Bias

A measurement artifact where an experimenter's expectations or hypotheses unintentionally bias data collection or scoring, controlled by blinding.

Demand Characteristics

Cues or features in a study that indicate the research hypothesis to participants, prompting them to change their authentic behavior.

Socially Desirable Responding

A form of participant reactivity where participants respond in ways they perceive as socially acceptable rather than reflecting their true feelings or actions.

Range Effects

A measurement limitation where observations cluster near the extremes of a measurement scale, inhibiting detection of true differences across participants.

Ceiling Effect

A range effect where scores cluster at the highest end of a scale due to a task or measurement being too easy.

Floor Effect

A range effect where scores cluster at the lowest end of a scale due to a task or measurement being too difficult.