insh6300 - week 2 (Measurement, Inference, and Statistical Power)

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/101

flashcard set

Earn XP

Description and Tags

A comprehensive vocabulary review deck covering measurement, conceptualization, operationalization, question wording, validity, reliability, sampling bias, statistical power, and academic publishing.

102 Terms

1
New cards

Describing a Phenomenon

A common research goal focused on depicting what is occurring, such as identifying which country has the happiest population.

2
New cards

Explaining a Phenomenon

A common research goal aimed at uncovering underlying reasons or causes, such as determining why people living in Nordic countries are happier than individuals from other countries.

3
New cards

Predicting a Phenomenon

A common research goal focused on using one variable to forecast another, such as evaluating how well a parent's happiness predicts the happiness of their children.

4
New cards

Evaluating a Program, Policy, or Practice

A research goal connected to explanation that determines whether an intervention produces an intended effect, such as testing if reading a specific self-help book increases one's happiness.

5
New cards

Self-Report Measurement

A data collection method in which participants report their own internal states, beliefs, or experiences through interviews or surveys.

6
New cards

Behavioral Observation Measurement

A measurement strategy where researchers directly observe subjects to systematically record observable actions, such as counting happy and unhappy behaviors.

7
New cards

Archival and Administrative Data

Pre-existing institutional or digital records used as measurement sources, such as mental health clinic records or Google search trends.

8
New cards

Quantitative Measurement

The practice of using numbers to evaluate and rank individuals or units according to a specific attribute, such as their level of happiness.

9
New cards

Qualitative Measurement

The practice of using words, images, sounds, and other non-numerical media to describe characteristics and differences in a phenomenon.

10
New cards

Measurement Vagueness

A questioning flaw where items lack temporal or situational specification, such as asking 'Are you happy?' without clarifying whether it refers to right now, over recent weeks, or across recent years.

11
New cards

Lack of Standardization (in Questioning)

A measurement defect where respondents do not share a uniform response format, causing some to assume only yes/no answers are allowed while others consider a wider range such as 'a little.'

12
New cards

Lack of Accuracy (in Questioning)

A measurement flaw occurring when a respondent's personal definition of a term differs from the researcher's or other respondents' definitions.

13
New cards

Lack of Precision (in Measurement)

An instrument's inability to distinguish between meaningfully different states, such as when someone feeling slightly down and someone who is clinically depressed both answer 'no' to being happy.

14
New cards

Low Measurement Reliability

A flaw where an instrument produces fluctuating results over repeated administrations because answers depend heavily on transient factors like mood at the time.

15
New cards

Good Measurement

A standardized procedure that translates an abstract construct into an observable representation that is accurate (valid), precise (sensitive), and reliable (consistent).

16
New cards

Construct

An abstract concept or theoretical idea that cannot be directly observed, such as happiness or political apathy.

17
New cards

Conceptualization

The process of defining an abstract construct by breaking it down and explicitly specifying its distinct features, parts, or dimensions.

18
New cards

Operationalization

The process of defining an abstract construct in terms of a concrete, measurable procedure or observable rule.

19
New cards

Measure (Measurement Instrument)

The specific tool, question, or approach used to systematically record unit differences in an operationalized construct.

20
New cards

Cantril Ladder

A visual measurement instrument where respondents rate their overall life satisfaction on rungs ranging from worst possible life (00) to best possible life (1010).

21
New cards

Resource-Constrained Hunger (Conceptualization)

Conceptualizing hunger specifically as physiological hunger brought on by a lack of money to purchase food, rather than mental or other causes.

22
New cards

Resource-Constrained Hunger (Operationalization and Measure)

Operationalizing hunger as the frequency of not eating when hungry due to financial constraints, measured using a 4-point scale: 1. Very often, 2. Sometimes, 3. Not very often, 4. Never.

23
New cards

Child Neglect (Conceptualization)

The conceptual definition of child neglect as the failure to provide adequate supervision and basic needs to a minor under one's guardianship.

24
New cards

Child Neglect (Operationalization and Measure)

Operationalizing neglect as instances of inadequate supervision (leaving a minor alone past age-appropriate limits) or basic needs failure (lacking shelter, clean clothing, heating/cooling, or food), recorded via specific behavioral screening items.

25
New cards

Political Apathy (Conceptualization)

Specifying political apathy through three conceptual features: participation in the political process, political thinking, and interest in politics.

26
New cards

Political Apathy (Operationalization)

Defining participation through acts like voting in the last election, protests, and boycotts; thinking through awareness of ongoing political events; and interest through frequency of discussing politics and reading news.

27
New cards

Climate Displacement/Migration (Conceptualization)

Conceptualizing climate displacement as temporarily or involuntarily moving from one's residence due to a natural disaster brought on by climate events.

28
New cards

Climate Displacement/Migration (Operationalization)

Defining involuntary moving as moving from one residence to another on a temporary basis out of force or necessity, and climate disaster as natural, earthly, or abnormal weather events.

29
New cards

Measurement Validity (Accuracy)

The ability of a measurement instrument to successfully and accurately capture the specific construct it is intended to measure.

30
New cards

Measurement Reliability (Consistency)

The ability of a measurement instrument to yield consistent, repeatable, and stable scores over time and across administrations.

31
New cards

Measurement Precision (Sensitivity)

The ability of a measurement instrument to detect small, fine-grained differences in the construct being evaluated.

32
New cards

Richness and Depth (Qualitative Research)

Important criteria in qualitative inquiry focused on obtaining comprehensive, detailed, and context-laden descriptions of a phenomenon.

33
New cards

Correlation Coefficient (in Reliability)

A standardized statistical index ranging from −1.0-1.0 to 1.01.0 that measures the strength and direction of the relationship between two measurement scores.

34
New cards

Weak Correlation Benchmark

A correlation coefficient magnitude falling within the range of 0.00.0 to ∣0.29∣|0.29|.

35
New cards

Moderate Correlation Benchmark

A correlation coefficient magnitude falling within the range of ∣0.30∣|0.30| to ∣0.69∣|0.69|.

36
New cards

Strong Correlation Benchmark

A correlation coefficient magnitude falling within the range of ∣0.70∣|0.70| to ∣1.0∣|1.0|.

37
New cards

Test-Retest Reliability

A type of reliability evaluated by calculating the correlation coefficient between scores obtained when the exact same measure is administered to the same individuals at different times.

38
New cards

Inter-Item Reliability (Internal Consistency)

An assessment of reliability evaluating whether multiple items in an instrument that are designed to measure the same construct produce consistent scores.

39
New cards

Cronbach's Alpha

A correlation coefficient used to quantify the internal consistency and inter-item reliability among multiple items intended to measure a common construct.

40
New cards

Inter-Rater Reliability

A correlation coefficient indicating the degree of agreement or consensus in measurements made independently by two or more observers.

41
New cards

Construct Validity

An overarching form of measurement validity that assesses the extent to which an operational measure accurately captures the intended theoretical construct.

42
New cards

Convergent Validity

A subtype of construct validity demonstrated by correlating scores on a target measure with a different validated scale designed to measure the same construct.

43
New cards

Divergent Validity

A subtype of construct validity demonstrated when scores on a measure show weak or negative correlations with a validated scale measuring an unrelated or opposite construct.

44
New cards

Criterion Validity

A form of measurement validity that assesses the extent to which a measure accurately predicts current or future behavior and observable outcomes.

45
New cards

Concurrent Validity

A subtype of criterion validity that evaluates how accurately a measure predicts or corresponds to a participant's current behavior.

46
New cards

Predictive Validity

A subtype of criterion validity that evaluates how accurately a measure forecasts a participant's future behavior or outcomes.

47
New cards

Content Validity

A non-statistical form of validity, judged by expert consensus, evaluating whether a set of measures captures the complete, multifaceted content of a construct.

48
New cards

Face Validity

A non-statistical judgment by experts evaluating whether a measurement instrument intuitively appears valid and appropriate at face value.

49
New cards

Measurement Invariance

The psychometric property ensuring that an instrument captures the identical construct with equal accuracy across diverse participant groups, ages, cultures, or time periods.

50
New cards

Averages vs. Variation

The methodological principle that aggregate group averages can obscure important internal variation, such as an unhappier country on average having individuals happier than a happier country's happiest.

51
New cards

Open-Ended Question

A question format that allows participants to construct answers entirely in their own words without being restricted to predetermined choices.

52
New cards

Closed-Ended Question

A question format that requires respondents to select an answer from a provided set of predetermined response options.

53
New cards

Likert Scale

A closed-ended response format that arranges varying levels of agreement, disagreement, or support for a statement along an ordered continuum.

54
New cards

Multiple-Choice Question

A closed-ended question format that prompts respondents to choose one or more appropriate items from a provided list.

55
New cards

Ranking Question

A survey question format requiring respondents to arrange a specified list of items into an ordered sequence based on a particular dimension.

56
New cards

Hybrid Question Format

A questionnaire item combining closed-ended choices with an open-ended field, such as a multiple-choice list ending with 'Other (please specify): _________.'

57
New cards

Fowler and Cosenza (2009)

The scholarly source cited for guidelines on sound question wording, including simplicity, avoiding embarrassment, limiting guessing, and ensuring mutually exclusive response options.

58
New cards

Exhaustive Response Options

A question-design requirement that every possible response choice is provided, ensuring every participant can find an applicable option, including 'don't know.'

59
New cards

Mutually Exclusive Response Options

A question-design standard dictating that response categories do not overlap, ensuring a respondent can select no more than one option.

60
New cards

Leading Question

A flawed question worded in a manner that predisposes respondents to answer a certain way by embedding an authoritative premise or suggesting a correct answer.

61
New cards

Double-Barreled Question

A questionnaire error where two separate questions are combined into a single sentence while asking for only one combined response.

62
New cards

Dichotomization

The poor measurement practice of collapsing a continuous or multi-category measure into a binary (two-category) variable, resulting in lost information and reduced statistical power.

63
New cards

Single-Item Measurement of Complex Constructs

The flawed practice of capturing a multifaceted construct with only one item, which frequently overlooks key dimensions and inflates vulnerability to measurement error.

64
New cards

Blind Scale Aggregation

The problematic practice of mechanically combining multiple distinct measures into a single scale without considering whether individual items should be weighted unequally.

65
New cards

Caprara et al. (2009)

A published empirical study discussed in lecture examining the conceptualization and measurement of perceived political self-efficacy.

66
New cards

Barbara Geddes (1990)

The scholar who established that the two crucial tasks in hypothesis testing are identifying the universe of cases to which the hypothesis applies and finding or developing variable measures.

67
New cards

Statistical Inference

The practice of using data from an observed sample along with statistical theory to estimate or infer values for an unobserved population.

68
New cards

Population

The entire overarching group of units or individuals that a researcher is interested in studying.

69
New cards

Sample

A selected subset of units (such as individuals, businesses, or countries) drawn directly from the larger population of interest.

70
New cards

Population Representation

The degree to which the aggregate distribution of characteristics within an observed sample collectively reflects that of the broader population.

71
New cards

Random Sampling

A sampling procedure where every single unit in a target population possesses an equal probability of being selected into the research sample.

72
New cards

Sample Selection Bias

Systematic bias that emerges because some members of a population are more likely to be selected into a sample than others.

73
New cards

Selection on the Dependent Variable

A major cause of selection bias occurring when cases are sampled based on whether they display the outcome of interest, truncating variation and distorting causal inference.

74
New cards

Survivorship Bias

A form of selection bias where researchers focus exclusively on entities that survived or succeeded through a selection process while ignoring failures that dropped out.

75
New cards

Internal Validity

The degree to which a study's design can convincingly prove that changes in the independent variable directly caused changes in the dependent variable.

76
New cards

External Validity (Generalizability)

The degree to which empirical findings based on a specific sample can be applied, extended, or generalized to other groups, settings, and contexts.

77
New cards

Ecological Fallacy

An inferential mistake committed when conclusions about smaller units of analysis (such as individuals) are inappropriately deduced from aggregate data of larger units (such as cities).

78
New cards

Unit of Analysis

The specific entity being studied and measured in a research project, such as individuals, organizations, or cities.

79
New cards

Statistical Power

The mathematical probability that a study will detect a real effect, equivalent to the probability of avoiding a Type 2 error.

80
New cards

Type 1 Error

A statistical decision error characterized as a 'false positive,' occurring when a researcher finds an effect that is not actually present in reality.

81
New cards

Type 2 Error

A statistical decision error characterized as a 'false negative,' occurring when a researcher fails to detect a real effect that truly exists.

82
New cards

Conventional Statistical Power Benchmark

The standard research benchmark of 80%80\% statistical power, representing an accepted 20%20\% chance of missing a real effect (committing a Type 2 error).

83
New cards

Minimal Detectable Effect (MDE)

The smallest true effect size that a study design has a good chance of reliably detecting given its sample size and statistical parameters.

84
New cards

Power Analysis

A statistical calculation conducted prior to data collection to determine the sample size required to detect an expected effect size at specified alpha and power levels.

85
New cards

Effect Size (in Statistical Power)

The magnitude of a true relationship or difference between variables; larger true effects are substantially easier for a study to detect.

86
New cards

Significance Threshold (Alpha, in Power)

The probability threshold for rejecting the null hypothesis; choosing a stricter alpha (such as 0.010.01 instead of 0.050.05) reduces statistical power.

87
New cards

One-Sided vs. Two-Sided Hypothesis Tests (in Power)

A test specification where directional one-sided hypothesis tests require smaller sample sizes to achieve equivalent power compared to two-sided tests.

88
New cards

Outcome Variance and Noise (in Power)

Unexplained variability in an outcome measure that makes detecting an effect harder; adding control variables that explain outcome variation improves statistical power.

89
New cards

Main Effects vs. Interaction Effects (in Power)

A design consideration where detecting an interaction effect between variables requires a much larger sample size than detecting a main effect.

90
New cards

Szucs and Ioannidis (2017)

A meta-scientific analysis reporting that cognitive neuroscience and psychology studies had an average power of 44%44\% for medium effects, resulting in a >50%>50\% false report probability.

91
New cards

Button et al. (2013)

A study analyzing 730730 neuroscience papers that found a median statistical power of 21%21\%, meaning studies had only a 1 in 51\text{ in }5 chance of detecting real effects (80%80\% false negative rate).

92
New cards

Ioannidis et al. (2017)

A meta-scientific investigation of 6,7006{,}700 economics studies revealing an alarmingly low median statistical power of 18%18\%.

93
New cards

Arel-Brundcok et al. (2026)

A research review analyzing approximately 2,0002{,}000 political science studies that identified a median statistical power of only 10%10\%, as cited in the lecture transcript.

94
New cards

Underpowered Studies

Studies conducted with insufficient sample sizes that run high risks of both missing real effects and exaggerating the magnitude of detected effects.

95
New cards

Publication Bias

The systemic tendency of academic journals to publish statistically significant, positive findings over null or non-significant results.

96
New cards

P-Hacking

The questionable research practice of manipulating data transformations, variable selections, or model specifications until non-significant results cross the statistical significance threshold.

97
New cards

HARKing

An acronym for 'Hypothesizing After the Results are Known,' describing the practice of presenting post-hoc findings as if they were a priori predictions.

98
New cards

Peer Review

A scientific vetting mechanism where one or more external scientists independently review, critique, and evaluate a study prior to journal publication.

99
New cards

Blinded Peer Review

A peer-review format in which the identities of the authors and reviewers are concealed from each other to prevent bias.

100
New cards

Subscription-Based vs. Open-Access Journals

Two academic publishing models: subscription journals require readers or institutions to pay access fees, whereas open-access journals charge authors fees to make articles freely available.