insh6300 - week 8 (Experimental Methods in the Social Sciences)

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/115

flashcard set

Earn XP

Description and Tags

A comprehensive set of 100 vocabulary flashcards covering experimental methods, causal inference, assignment mechanisms, validity threats, statistical estimation, and real-world social science field experiments.

116 Terms

1
New cards

Causality (Three Requirements)

The three criteria required to establish a causal relationship: the cause and effect covary, the cause occurs before the effect in time, and all alternative explanations for the relationship can be ruled out.

2
New cards

Covariation

A condition for causality in which the proposed cause and effect change together—when one variable changes, so does the other.

3
New cards

Temporal Precedence

The requirement in causal inference that the independent variable (cause) must occur prior in time to the dependent variable (effect).

4
New cards

Ruling Out Alternative Explanations

The requirement in causal inference to eliminate plausible rival hypotheses, third variables, or confounders that could account for an observed association.

5
New cards

Longitudinal Study (re: Causality)

A study that follows subjects over time, noted as particularly helpful for identifying covariates and establishing the proper temporal order of cause and effect.

6
New cards

Internal Validity

The extent to which a study design can rule out alternative explanations for an observed association between variables.

7
New cards

Counterfactual World

A hypothetical, identical parallel world where the proposed cause does not change, used as the benchmark to determine what would have happened to the outcome in the absence of the treatment.

8
New cards

Approximating the Counterfactual

The process by which experimental researchers use a control group combined with random assignment to simulate the unobservable counterfactual world.

9
New cards

Fundamental Problem of Causal Inference

The methodological barrier that a researcher cannot simultaneously observe the same exact unit (e.g., a family) receiving a treatment and not receiving that treatment at the exact same time.

10
New cards

Experiment

An empirical investigation in which the independent variable is under the direct control of the investigator.

11
New cards

Randomized Controlled Trial (RCT)

An experimental investigation that uses random assignment to allocate units to different values or conditions of the independent variable.

12
New cards

Observational Study

A study design where the allocation of individuals to different values of the independent variable is entirely outside of the investigator's control.

13
New cards

Laboratory Experiment

An experiment carried out in an artificial, highly controlled setting created by the researcher.

14
New cards

Field Experiment

An experiment carried out in a natural, real-world setting where units typically live or work.

15
New cards

Simulated Experiment

A computational research approach where researchers construct a computer model of a system and systematically test conditions without randomly assigning human subjects.

16
New cards

Random Assignment

The application of a random chance procedure to assign units to different values, conditions, or treatments of the independent variable.

17
New cards

Treatment Group

The set of units in an experiment that receives a specific intervention, active condition, or dosage of the independent variable.

18
New cards

Control Group

The group of units in an experiment that serves as the baseline standard of comparison to assess the impact of treatments, representing the best estimate of the counterfactual.

19
New cards

Placebo Group

A control group that receives an inert substance or inactive condition to benchmark against active treatment groups.

20
New cards

Known, Nonzero Probability

The property of random assignment guaranteeing that every unit in the study has an identified probability greater than zero of receiving each possible value of the independent variable.

21
New cards

Equivalence in Expectation

The theoretical principle that random assignment distributes all unit differences—both observed and unobserved—equally across treatment conditions on average.

22
New cards

Pre-treatment Factors

Baseline unit characteristics and background variables that exist prior to the administration of an experimental treatment, which random assignment balances in expectation.

23
New cards

Chance Differences

Random imbalances between experimental groups that occur by bad luck in any single experiment, which become smaller and less probable as the sample size increases.

24
New cards

Self-Selection

A non-experimental process where individuals choose for themselves whether to participate in or receive an intervention, introducing confounding variables.

25
New cards

Confounder

A third variable that is associated with both the independent variable (e.g., enrollment) and the dependent variable (e.g., job placement), distorting the apparent causal relationship.

26
New cards

Motivation (as a Confounder)

An unobserved characteristic in the job training example that simultaneously increases an individual's likelihood of enrolling in training and their likelihood of securing employment.

27
New cards

Selection Bias

A systematic difference between groups arising because individuals who select into a treatment differ fundamentally from those who do not, independent of the treatment effect.

28
New cards

Selection Bias Gap (Job Training Example)

The 4545 percentage point difference (70%70\% vs. 25%25\%) observed under self-selection, where only 1515 points are caused by the program and 3030 points stem from motivation bias.

29
New cards

Baseline Equivalence

A state achieved under random assignment where both treatment and control groups share identical initial expected outcomes (e.g., both starting at 40%40\% job attainment without the program).

30
New cards

Average Treatment Effect (ATE)

The average change in the dependent variable caused by the independent variable (treatment) across all units in the experimental sample.

31
New cards

Difference-in-Means ATE Estimator

The standard calculation of the ATE obtained by subtracting the average outcome of the control group from the average outcome of the treatment group.

32
New cards

High Dosage vs. Placebo ATE (Blood Pressure Example)

The calculated average treatment effect of 82−126.5=−44.582 - 126.5 = -44.5, indicating the high dose pill lowered blood pressure by 44.544.5 points on average compared to placebo.

33
New cards

Low Dosage vs. Placebo ATE (Blood Pressure Example)

The calculated average treatment effect of 98−126.5=−28.598 - 126.5 = -28.5, indicating the low dose pill lowered blood pressure by 28.528.5 points on average compared to placebo.

34
New cards

High Dosage vs. Low Dosage ATE (Blood Pressure Example)

The calculated average treatment effect of 82−98=−1682 - 98 = -16, demonstrating that randomized designs permit direct causal comparisons between different active treatment conditions.

35
New cards

Unbiased Estimate

An experimental estimate produced when the treatment is the only systematic difference between the experimental groups, centering on the true value across repeated trials.

36
New cards

Bias

A systematic difference between groups, other than the treatment, that pushes the treatment effect estimate in the same direction every time an experiment is repeated.

37
New cards

Imprecision (in ATE Estimates)

Estimation error caused by random chance differences scattering single experiment results around the true bullseye, which shrinks as the sample size increases.

38
New cards

Confidence Interval (ATE)

A statistical range around the estimated treatment effect that quantifies the degree of imprecision and sampling uncertainty.

39
New cards

Nonprobability Sampling (Sampson 2010)

A sampling approach (e.g., relying on volunteers, court defendants, or prisoners) that likely biases the ATE for the broader population when treatment effects vary across units.

40
New cards

Between-Unit Study

A study design that estimates treatment effects by comparing outcomes across different groups of units—those assigned to receive the treatment versus those assigned to control.

41
New cards

Within-Unit Study

A study design where each individual unit receives multiple conditions over time, eliminating the confounding impact of stable unit differences because each unit serves as its own control.

42
New cards

Lasting Treatment Effects

Outcomes that permanently alter participants (e.g., job training skills, moving neighborhoods, or therapy) and cannot be undone, requiring a between-unit design.

43
New cards

Washout Period

A designated period of time left between successive conditions in a within-unit design to ensure the effects of an earlier treatment completely dissipate before the next begins.

44
New cards

Carryover Effect

A primary risk in within-unit studies where exposure to an earlier treatment condition alters or confounds the participant's response to subsequent conditions.

45
New cards

Participant Fatigue

A threat in within-unit designs where subjects become tired, bored, or less engaged as they undergo multiple consecutive experimental conditions.

46
New cards

Condition Order Randomization

A methodological solution in within-unit studies where the sequential order of experimental conditions is randomized across subjects to mitigate order effects.

47
New cards

Assignment Mechanism

The specific procedural rule or process that determines which experimental units receive which values of the independent variable.

48
New cards

Bernoulli Trial

A randomized assignment mechanism where each unit is assigned independently with a fixed probability (often 0.50.5), like a coin flip, resulting in variable group sizes.

49
New cards

Dynamic Arrival Setting

A research context where units arrive over time rather than being available all at once in a pre-existing sampling frame, frequently utilizing Bernoulli trials.

50
New cards

Completely Randomized Experiment

An experimental assignment mechanism where a fixed total number of units from an available pool is assigned to receive treatment or control with equal probability.

51
New cards

Propensity Score

The probability that a given unit is assigned to the treatment condition, which is uniform (Ntreatment/NN_{\text{treatment}} / N) for all units in a completely randomized experiment.

52
New cards

Stratified / Block Randomized Experiment

An experimental design where units are first partitioned into homogeneous subgroups (strata/blocks) based on covariates, followed by a completely randomized experiment within each block.

53
New cards

Strata / Blocks

Subsets of experimental units grouped together prior to randomization based on shared baseline characteristics (e.g., biological sex, age) that predict the outcome.

54
New cards

Blocking Variable

A pre-treatment covariate that strongly predicts the outcome measure and is used to partition units into blocks to eliminate chance imbalances.

55
New cards

Paired Randomized Experiment

An assignment design where units are matched into blocks of two based on predictive covariates, and exactly one unit in each pair is randomly assigned to treatment.

56
New cards

Small Sample Imbalance (in Bernoulli Trials)

The substantial probability in small samples of obtaining severe group size splits by chance (e.g., flipping 1010 coins yields an 8/28/2 split approximately 9%9\% of the time and a 7/37/3 split approximately 23%23\% of the time).

57
New cards

Pairwise Attrition Cost

A vulnerability of paired randomized designs where the dropout of one unit may force the researcher to discard its matched partner, substantially reducing sample size.

58
New cards

Clustering

The practice of aggregating individual units into larger social or geographic groups (e.g., classrooms, lines, neighborhoods) and assigning treatment at the group level.

59
New cards

Cluster-Level Analysis

The statistical requirement in cluster-randomized trials to analyze data at the cluster level, recognizing that the total number of clusters dictates precision more than unit count.

60
New cards

Precision Loss in Clustering

The reduction in statistical power and precision associated with cluster-randomized designs compared to individual-level random assignment due to correlated unit behaviors.

61
New cards

Multi-Level Experiment

An experimental design in which random assignment occurs across multiple hierarchical tiers, such as assigning lines to procedures and then assigning individual passengers within those lines.

62
New cards

Hierarchical Interaction

The statistical interaction estimated in multi-level experiments examining whether the effect of an individual-level treatment depends on the cluster-level treatment condition.

63
New cards

Treatment Effect Heterogeneity

Variations in how different experimental units or demographic subgroups respond to the identical treatment intervention.

64
New cards

Hidden Treatment Effects

A situation where an aggregate Average Treatment Effect (ATE) of zero obscures substantial positive effects for some participants and negative effects for others.

65
New cards

Subgroup Analysis

The analytical practice of estimating the ATE separately within distinct categories of participants (e.g., comparing males vs. females).

66
New cards

Observational Nature of Subgroups

The methodological limitation that subgroup membership is not randomly assigned, meaning differing subgroup ATEs could be driven by underlying confounders rather than the subgroup trait.

67
New cards

Factorial Design

An experimental design that randomly assigns units to every possible combination of two or more treatments to test separate main effects and interaction effects simultaneously.

68
New cards

3×33 \times 3 Factorial Experiment (Slide Example)

A specific design testing medication dose (high dosage, low dosage, placebo) crossed with exercise intensity (high intensity, low intensity, no exercise).

69
New cards

Causal Interaction

A finding derived from a factorial design demonstrating that the causal impact of one randomly assigned variable depends upon the assigned level of another variable.

70
New cards

Waitlist / Stepped-Wedge Design

A design where all units begin in the control condition and cross over into the treatment group at staggered, randomly assigned calendar intervals.

71
New cards

Secular Trends (in Stepped-Wedge Designs)

Outside real-world events or time-related historical changes that coincide with treatment roll-out and can easily be mistaken for treatment effects.

72
New cards

Adequate Sample Size

A necessary ingredient for experiments ensuring that chance differences average out and statistical uncertainty surrounding the ATE is sufficiently minimized.

73
New cards

Rare Outcomes Challenge

A methodological issue where an outcome occurs very infrequently, requiring a massive sample size to detect statistically meaningful differences between groups.

74
New cards

Compliance (as an Ingredient)

The requirement that experimental participants actually receive and adhere to the specific treatment condition to which they were randomly assigned.

75
New cards

Interference / Contamination

A breakdown in experimental isolation occurring when units in one treatment group experience or receive aspects of another group's assigned condition.

76
New cards

Differential Attrition

The systematic or unequal dropout of participants across treatment arms that undermines baseline equivalence and biases the ATE estimate.

77
New cards

Excludability

The foundational assumption that assignment to a treatment group affects the dependent variable solely through the intended active mechanism of the assigned treatment.

78
New cards

Hawthorne Effect

An excludability violation where subjects alter their natural behavior simply because they know they are being observed or evaluated by researchers.

79
New cards

John Henry Effect

An excludability violation where control group members alter their behavior to work harder or overcome their perceived disadvantage relative to the treatment group.

80
New cards

Staff Expectancy Violation

An excludability violation occurring when researchers or program staff inadvertently treat subjects differently because they know their assigned group conditions.

81
New cards

Time-Varying Confounder

An extraneous historical event occurring during the experimental window that impacts one experimental condition more than another.

82
New cards

Measurement Consistency

The practice of using valid, reliable, and standardized outcome instruments administered in the identical manner across all experimental arms.

83
New cards

Noncompliance

A condition where units assigned to a specific value of the independent variable end up receiving an alternative condition instead.

84
New cards

One-Sided Noncompliance

A scenario where noncompliance is restricted to the treatment arm, such that assigned treatment units can refuse or quit, but control units cannot access the treatment.

85
New cards

Two-Sided Noncompliance

A scenario where units assigned to the control group manage to obtain the treatment and units assigned to the treatment group fail to receive it.

86
New cards

Intention-to-Treat (ITT) / Treatment as Assigned

An analytical approach that estimates the treatment effect by analyzing subjects strictly within the groups to which they were originally randomized, regardless of whether they complied.

87
New cards

Treatment as Delivered

An analysis comparing outcomes based on the actual condition subjects received rather than what they were assigned, which destroys randomized equivalence and introduces confounding.

88
New cards

Dilution of Effect (in ITT)

The downward attenuation of the estimated treatment effect under ITT analysis caused by including noncompliant participants who never actually received the treatment.

89
New cards

Policy Relevance of ITT

The practical value of an ITT estimate in reflecting the real-world net impact of offering an intervention to a general population where adherence is not universal.

90
New cards

Stable Unit Treatment Value Assumption (SUTVA)

The core assumption that each unit's outcome depends strictly on its own treatment assignment, and that all units within a treatment group receive identical treatment.

91
New cards

SUTVA Part 1 (No Interference)

The specific requirement that one unit's experimental outcome is completely unaffected by the treatment assignments or actions of any other units.

92
New cards

SUTVA Part 2 (Uniform Treatment Delivery)

The specific requirement that every unit within a given treatment group receives the exact same dosage and standardized implementation of the intervention.

93
New cards

Spillover Bias Toward Zero

The tendency of SUTVA violations to underestimate treatment efficacy because program benefits leaking to control units make both groups appear more similar.

94
New cards

Displacement of Harms

A SUTVA violation where an intervention diverts negative behaviors (e.g., crime or accidents) into adjacent control zones, artificially exaggerating the perceived treatment effect.

95
New cards

Buffer Areas

Geographic safety zones deliberately left between treatment and control territories to prevent physical spillover and protect SUTVA.

96
New cards

Random Attrition

Participant dropout that occurs purely by chance and at equal rates across groups, causing a loss of sample size and precision without inducing systematic bias.

97
New cards

Oversampling (for Attrition)

Recruiting and randomizing more participants than mathematically required based on expected dropout rates to safeguard post-attrition statistical power.

98
New cards

Bounding / Trimming (Attrition Strategy)

Analytic techniques that impute extreme potential values or trim sample distributions based on baseline covariates to calculate the upper and lower bounds of the true ATE.

99
New cards

Administrative Records Tracking

Using official, non-reactive records (e.g., arrest logs, tax records) to measure outcomes directly without relying on participant contact, mitigating attrition.

100
New cards

Implementation Evaluation

An assessment conducted alongside an experiment to confirm that treatments were delivered with fidelity across sites/staff and to identify time-varying confounders.