1/101
A comprehensive vocabulary review deck covering measurement, conceptualization, operationalization, question wording, validity, reliability, sampling bias, statistical power, and academic publishing.
Describing a Phenomenon
A common research goal focused on depicting what is occurring, such as identifying which country has the happiest population.
Explaining a Phenomenon
A common research goal aimed at uncovering underlying reasons or causes, such as determining why people living in Nordic countries are happier than individuals from other countries.
Predicting a Phenomenon
A common research goal focused on using one variable to forecast another, such as evaluating how well a parent's happiness predicts the happiness of their children.
Evaluating a Program, Policy, or Practice
A research goal connected to explanation that determines whether an intervention produces an intended effect, such as testing if reading a specific self-help book increases one's happiness.
Self-Report Measurement
A data collection method in which participants report their own internal states, beliefs, or experiences through interviews or surveys.
Behavioral Observation Measurement
A measurement strategy where researchers directly observe subjects to systematically record observable actions, such as counting happy and unhappy behaviors.
Archival and Administrative Data
Pre-existing institutional or digital records used as measurement sources, such as mental health clinic records or Google search trends.
Quantitative Measurement
The practice of using numbers to evaluate and rank individuals or units according to a specific attribute, such as their level of happiness.
Qualitative Measurement
The practice of using words, images, sounds, and other non-numerical media to describe characteristics and differences in a phenomenon.
Measurement Vagueness
A questioning flaw where items lack temporal or situational specification, such as asking 'Are you happy?' without clarifying whether it refers to right now, over recent weeks, or across recent years.
Lack of Standardization (in Questioning)
A measurement defect where respondents do not share a uniform response format, causing some to assume only yes/no answers are allowed while others consider a wider range such as 'a little.'
Lack of Accuracy (in Questioning)
A measurement flaw occurring when a respondent's personal definition of a term differs from the researcher's or other respondents' definitions.
Lack of Precision (in Measurement)
An instrument's inability to distinguish between meaningfully different states, such as when someone feeling slightly down and someone who is clinically depressed both answer 'no' to being happy.
Low Measurement Reliability
A flaw where an instrument produces fluctuating results over repeated administrations because answers depend heavily on transient factors like mood at the time.
Good Measurement
A standardized procedure that translates an abstract construct into an observable representation that is accurate (valid), precise (sensitive), and reliable (consistent).
Construct
An abstract concept or theoretical idea that cannot be directly observed, such as happiness or political apathy.
Conceptualization
The process of defining an abstract construct by breaking it down and explicitly specifying its distinct features, parts, or dimensions.
Operationalization
The process of defining an abstract construct in terms of a concrete, measurable procedure or observable rule.
Measure (Measurement Instrument)
The specific tool, question, or approach used to systematically record unit differences in an operationalized construct.
Cantril Ladder
A visual measurement instrument where respondents rate their overall life satisfaction on rungs ranging from worst possible life (0) to best possible life (10).
Resource-Constrained Hunger (Conceptualization)
Conceptualizing hunger specifically as physiological hunger brought on by a lack of money to purchase food, rather than mental or other causes.
Resource-Constrained Hunger (Operationalization and Measure)
Operationalizing hunger as the frequency of not eating when hungry due to financial constraints, measured using a 4-point scale: 1. Very often, 2. Sometimes, 3. Not very often, 4. Never.
Child Neglect (Conceptualization)
The conceptual definition of child neglect as the failure to provide adequate supervision and basic needs to a minor under one's guardianship.
Child Neglect (Operationalization and Measure)
Operationalizing neglect as instances of inadequate supervision (leaving a minor alone past age-appropriate limits) or basic needs failure (lacking shelter, clean clothing, heating/cooling, or food), recorded via specific behavioral screening items.
Political Apathy (Conceptualization)
Specifying political apathy through three conceptual features: participation in the political process, political thinking, and interest in politics.
Political Apathy (Operationalization)
Defining participation through acts like voting in the last election, protests, and boycotts; thinking through awareness of ongoing political events; and interest through frequency of discussing politics and reading news.
Climate Displacement/Migration (Conceptualization)
Conceptualizing climate displacement as temporarily or involuntarily moving from one's residence due to a natural disaster brought on by climate events.
Climate Displacement/Migration (Operationalization)
Defining involuntary moving as moving from one residence to another on a temporary basis out of force or necessity, and climate disaster as natural, earthly, or abnormal weather events.
Measurement Validity (Accuracy)
The ability of a measurement instrument to successfully and accurately capture the specific construct it is intended to measure.
Measurement Reliability (Consistency)
The ability of a measurement instrument to yield consistent, repeatable, and stable scores over time and across administrations.
Measurement Precision (Sensitivity)
The ability of a measurement instrument to detect small, fine-grained differences in the construct being evaluated.
Richness and Depth (Qualitative Research)
Important criteria in qualitative inquiry focused on obtaining comprehensive, detailed, and context-laden descriptions of a phenomenon.
Correlation Coefficient (in Reliability)
A standardized statistical index ranging from −1.0 to 1.0 that measures the strength and direction of the relationship between two measurement scores.
Weak Correlation Benchmark
A correlation coefficient magnitude falling within the range of 0.0 to ∣0.29∣.
Moderate Correlation Benchmark
A correlation coefficient magnitude falling within the range of ∣0.30∣ to ∣0.69∣.
Strong Correlation Benchmark
A correlation coefficient magnitude falling within the range of ∣0.70∣ to ∣1.0∣.
Test-Retest Reliability
A type of reliability evaluated by calculating the correlation coefficient between scores obtained when the exact same measure is administered to the same individuals at different times.
Inter-Item Reliability (Internal Consistency)
An assessment of reliability evaluating whether multiple items in an instrument that are designed to measure the same construct produce consistent scores.
Cronbach's Alpha
A correlation coefficient used to quantify the internal consistency and inter-item reliability among multiple items intended to measure a common construct.
Inter-Rater Reliability
A correlation coefficient indicating the degree of agreement or consensus in measurements made independently by two or more observers.
Construct Validity
An overarching form of measurement validity that assesses the extent to which an operational measure accurately captures the intended theoretical construct.
Convergent Validity
A subtype of construct validity demonstrated by correlating scores on a target measure with a different validated scale designed to measure the same construct.
Divergent Validity
A subtype of construct validity demonstrated when scores on a measure show weak or negative correlations with a validated scale measuring an unrelated or opposite construct.
Criterion Validity
A form of measurement validity that assesses the extent to which a measure accurately predicts current or future behavior and observable outcomes.
Concurrent Validity
A subtype of criterion validity that evaluates how accurately a measure predicts or corresponds to a participant's current behavior.
Predictive Validity
A subtype of criterion validity that evaluates how accurately a measure forecasts a participant's future behavior or outcomes.
Content Validity
A non-statistical form of validity, judged by expert consensus, evaluating whether a set of measures captures the complete, multifaceted content of a construct.
Face Validity
A non-statistical judgment by experts evaluating whether a measurement instrument intuitively appears valid and appropriate at face value.
Measurement Invariance
The psychometric property ensuring that an instrument captures the identical construct with equal accuracy across diverse participant groups, ages, cultures, or time periods.
Averages vs. Variation
The methodological principle that aggregate group averages can obscure important internal variation, such as an unhappier country on average having individuals happier than a happier country's happiest.
Open-Ended Question
A question format that allows participants to construct answers entirely in their own words without being restricted to predetermined choices.
Closed-Ended Question
A question format that requires respondents to select an answer from a provided set of predetermined response options.
Likert Scale
A closed-ended response format that arranges varying levels of agreement, disagreement, or support for a statement along an ordered continuum.
Multiple-Choice Question
A closed-ended question format that prompts respondents to choose one or more appropriate items from a provided list.
Ranking Question
A survey question format requiring respondents to arrange a specified list of items into an ordered sequence based on a particular dimension.
Hybrid Question Format
A questionnaire item combining closed-ended choices with an open-ended field, such as a multiple-choice list ending with 'Other (please specify): _________.'
Fowler and Cosenza (2009)
The scholarly source cited for guidelines on sound question wording, including simplicity, avoiding embarrassment, limiting guessing, and ensuring mutually exclusive response options.
Exhaustive Response Options
A question-design requirement that every possible response choice is provided, ensuring every participant can find an applicable option, including 'don't know.'
Mutually Exclusive Response Options
A question-design standard dictating that response categories do not overlap, ensuring a respondent can select no more than one option.
Leading Question
A flawed question worded in a manner that predisposes respondents to answer a certain way by embedding an authoritative premise or suggesting a correct answer.
Double-Barreled Question
A questionnaire error where two separate questions are combined into a single sentence while asking for only one combined response.
Dichotomization
The poor measurement practice of collapsing a continuous or multi-category measure into a binary (two-category) variable, resulting in lost information and reduced statistical power.
Single-Item Measurement of Complex Constructs
The flawed practice of capturing a multifaceted construct with only one item, which frequently overlooks key dimensions and inflates vulnerability to measurement error.
Blind Scale Aggregation
The problematic practice of mechanically combining multiple distinct measures into a single scale without considering whether individual items should be weighted unequally.
Caprara et al. (2009)
A published empirical study discussed in lecture examining the conceptualization and measurement of perceived political self-efficacy.
Barbara Geddes (1990)
The scholar who established that the two crucial tasks in hypothesis testing are identifying the universe of cases to which the hypothesis applies and finding or developing variable measures.
Statistical Inference
The practice of using data from an observed sample along with statistical theory to estimate or infer values for an unobserved population.
Population
The entire overarching group of units or individuals that a researcher is interested in studying.
Sample
A selected subset of units (such as individuals, businesses, or countries) drawn directly from the larger population of interest.
Population Representation
The degree to which the aggregate distribution of characteristics within an observed sample collectively reflects that of the broader population.
Random Sampling
A sampling procedure where every single unit in a target population possesses an equal probability of being selected into the research sample.
Sample Selection Bias
Systematic bias that emerges because some members of a population are more likely to be selected into a sample than others.
Selection on the Dependent Variable
A major cause of selection bias occurring when cases are sampled based on whether they display the outcome of interest, truncating variation and distorting causal inference.
Survivorship Bias
A form of selection bias where researchers focus exclusively on entities that survived or succeeded through a selection process while ignoring failures that dropped out.
Internal Validity
The degree to which a study's design can convincingly prove that changes in the independent variable directly caused changes in the dependent variable.
External Validity (Generalizability)
The degree to which empirical findings based on a specific sample can be applied, extended, or generalized to other groups, settings, and contexts.
Ecological Fallacy
An inferential mistake committed when conclusions about smaller units of analysis (such as individuals) are inappropriately deduced from aggregate data of larger units (such as cities).
Unit of Analysis
The specific entity being studied and measured in a research project, such as individuals, organizations, or cities.
Statistical Power
The mathematical probability that a study will detect a real effect, equivalent to the probability of avoiding a Type 2 error.
Type 1 Error
A statistical decision error characterized as a 'false positive,' occurring when a researcher finds an effect that is not actually present in reality.
Type 2 Error
A statistical decision error characterized as a 'false negative,' occurring when a researcher fails to detect a real effect that truly exists.
Conventional Statistical Power Benchmark
The standard research benchmark of 80% statistical power, representing an accepted 20% chance of missing a real effect (committing a Type 2 error).
Minimal Detectable Effect (MDE)
The smallest true effect size that a study design has a good chance of reliably detecting given its sample size and statistical parameters.
Power Analysis
A statistical calculation conducted prior to data collection to determine the sample size required to detect an expected effect size at specified alpha and power levels.
Effect Size (in Statistical Power)
The magnitude of a true relationship or difference between variables; larger true effects are substantially easier for a study to detect.
Significance Threshold (Alpha, in Power)
The probability threshold for rejecting the null hypothesis; choosing a stricter alpha (such as 0.01 instead of 0.05) reduces statistical power.
One-Sided vs. Two-Sided Hypothesis Tests (in Power)
A test specification where directional one-sided hypothesis tests require smaller sample sizes to achieve equivalent power compared to two-sided tests.
Outcome Variance and Noise (in Power)
Unexplained variability in an outcome measure that makes detecting an effect harder; adding control variables that explain outcome variation improves statistical power.
Main Effects vs. Interaction Effects (in Power)
A design consideration where detecting an interaction effect between variables requires a much larger sample size than detecting a main effect.
Szucs and Ioannidis (2017)
A meta-scientific analysis reporting that cognitive neuroscience and psychology studies had an average power of 44% for medium effects, resulting in a >50% false report probability.
Button et al. (2013)
A study analyzing 730 neuroscience papers that found a median statistical power of 21%, meaning studies had only a 1 in 5 chance of detecting real effects (80% false negative rate).
Ioannidis et al. (2017)
A meta-scientific investigation of 6,700 economics studies revealing an alarmingly low median statistical power of 18%.
Arel-Brundcok et al. (2026)
A research review analyzing approximately 2,000 political science studies that identified a median statistical power of only 10%, as cited in the lecture transcript.
Underpowered Studies
Studies conducted with insufficient sample sizes that run high risks of both missing real effects and exaggerating the magnitude of detected effects.
Publication Bias
The systemic tendency of academic journals to publish statistically significant, positive findings over null or non-significant results.
P-Hacking
The questionable research practice of manipulating data transformations, variable selections, or model specifications until non-significant results cross the statistical significance threshold.
HARKing
An acronym for 'Hypothesizing After the Results are Known,' describing the practice of presenting post-hoc findings as if they were a priori predictions.
Peer Review
A scientific vetting mechanism where one or more external scientists independently review, critique, and evaluate a study prior to journal publication.
Blinded Peer Review
A peer-review format in which the identities of the authors and reviewers are concealed from each other to prevent bias.
Subscription-Based vs. Open-Access Journals
Two academic publishing models: subscription journals require readers or institutions to pay access fees, whereas open-access journals charge authors fees to make articles freely available.