THE FINAL RESEARCH METHODS

Which research design is best for a single treatment with full compliance?

Two-arm RCT

What is the best design to test multiple versions of a treatment?

Multi-arm experiment

Which design evaluates combined effects or interactions between treatments?

Factorial experiment

What design is used when treatment is rolled out gradually until all units are treated?

Stepped-wedge

Which design is appropriate for cases of noncompliance or when only encouragement is assigned?

IV / encouragement design

What design should be used when treatment is assigned based on a specific cutoff or threshold?

RDD (Regression Discontinuity Design)

What is the best design for asking sensitive survey questions to reduce lying?

List experiment

Which design detects discrimination using fake applications or identical CVs?

Audit experiment

What design evaluates multidimensional preferences by showing profiles with many attributes?

Conjoint experiment

Which design is used to study spillovers or peer effects within a group?

Saturation design

Which design identifies heterogeneous effects within specific subgroups?

Subgroup / CATE design

Define Internal Validity.

The degree to which we can trust the causal claim inside a specific study.

Define External Validity.

The degree to which results can be generalized beyond the study to other contexts.

What is the main difference between internal and external validity?

Internal is about the effect in the study; External is about the effect beyond the study.

Define Descriptive Inquiry.

An inquiry that observes and describes reality without using a counterfactual.

Define Causal Inquiry.

An inquiry that seeks to estimate effects by comparing potential outcomes (needs a counterfactual).

Define an Observational study.

A study where nature or the world assigns treatment.

Define an Experimental study.

A study where the researcher assigns treatment.

What is the "Counterfactual"?

What would have happened to a unit if they had received a different treatment status.

What is the "Fundamental Problem of Causal Inference"?

We can only observe one potential outcome for each unit at a time.

What does the "M" in the MIDA framework stand for?

Model (a representation of how the world works).

What does the "I" in the MIDA framework stand for?

Inquiry (the specific research question being asked).

What does the "D" in the MIDA framework stand for?

Data Strategy (how data is collected and measured).

What does the "A" in the MIDA framework stand for?

Answer Strategy (how the answer is estimated from the data).

What elements are included in a Data Strategy?

Sampling, measurement, attrition, missing data, and index construction.

What elements are included in a Design Strategy?

Random assignment, blocking, clustering, and treatment conditions.

What elements are included in an Answer Strategy?

Estimator, standard errors, confidence intervals, and regression.

What are the three components of a Model signature?

The variables in the model, their ranges, and their exogeneity/endogeneity.

Define Parametric Models.

Models that contain many assumptions about the mathematical nature of the relationship between variables (e.g., assuming it is linear).

Define Non-Parametric Models.

Models that avoid assumptions about mathematical form, stating only that a relationship exists (Design Agnostically).

Define Independent Variable (Treatment).

The variable that causes the outcome (often denoted as D or X).

Define Dependent Variable (Outcome).

The result or effect we care about explaining (denoted as Y).

What is a Confounder?

A variable that causes both the treatment and the outcome.

How should you handle a confounder in a study?

You should adjust/control for it to close backdoor paths.

Define a Mediator.

A variable that lies on the causal path between treatment and outcome.

What is a Collider?

A variable that is caused by both the treatment and the outcome.

Why should you NEVER adjust for a collider?

Controlling it opens a backdoor path and creates bias.

Define an Instrumental Variable (IV).

A variable that affects treatment but only affects outcome through the treatment.

What is a "Backdoor Path"?

A non-causal path between treatment and outcome created by confounders.

Define an Exogenous variable.

A variable not caused by other variables in the model (it comes from outside).

Which variable type is usually endogenous in political science?

Outcome variables.

Which variable type MUST be exogenous to be valid?

Instrumental variables.

What does the absence of nodes in a DAG represent?

The absence of all common causes (observed and unobserved) for any pair of variables.

Which measure of central tendency is not affected by outliers?

The Median.

Define the Mode.

The value in a distribution that occurs most frequently.

What is Covariance?

A measure of how much two variables change together.

What is a Linear Predictor?

A "line of best fit" that is mathematically closest to the observed data points.

Define Population.

The full group the researcher cares about (e.g., all Irish voters).

Define Sample.

The subset of the population actually studied.

What is Simple Random Sampling?

Each unit independently has a fixed probability of being included.

What is Complete Random Sampling?

Randomly selecting a fixed number of units from the population.

What is Stratified Sampling?

Randomly sampling within specific groups or strata (e.g., 50 men, 50 women).

What is Cluster Sampling?

Sampling entire groups or clusters (e.g., villages) rather than individuals.

Define Multistage Sampling.

Selecting clusters first, then selecting individuals within those clusters.

What is a Block Design?

Randomizing treatment within specific groups to increase balance.

What is the main goal of a Block Design?

To increase balance and precision.

What is the main goal of a Cluster Design?

To reduce costs, manage logistics, and prevent spillovers.

How does a Cluster Design affect variance?

It generally increases variance compared to individual randomization.

What does the Intra-Cluster Correlation Coefficient (ICC) measure?

How similar units are within the same cluster.

What does ICC = 0 imply?

Units in a cluster are independent (all variation is individual-level).

What does ICC = 1 imply?

Units in a cluster are identical (all variation is cluster-level).

What is Convenience Sampling?

Sampling units based on ease of access (e.g., people in a specific mall); usually results in high bias.

What is Purposive Sampling?

Selecting a sample based on the researcher's judgment of who will be most useful/representative.

What is Quota Sampling?

Selecting a sample by "type" (like stratified sampling) but without random selection.

Define Respondent-Driven (Snowball) Sampling.

A non-random method where initial subjects refer the researcher to other subjects.

Define Noncompliance

When units do not follow their assigned treatment status.

Define Attrition.

When units drop out of a study before it is finished.

What is a SUTVA violation?

When spillovers or multiple versions of treatment exist, violating the assumption that one unit's treatment doesn't affect another.

Define a "Complier".

A unit that takes treatment only if encouraged/assigned to do so.

Define an "Always-taker".

A unit that always takes treatment, regardless of their assignment.

Define a "Never-taker".

A unit that never takes treatment, regardless of their assignment.

Define a "Defier".

A unit that does the exact opposite of what they are assigned to do.

What does LATE stand for?

Local Average Treatment Effect.

For which group does IV identify the LATE?

Compliers.

What are the two groups in a Two-Arm Randomized Experiment?

Treatment group and Control group.

When is a Two-Arm Randomized Experiment the best choice?

One treatment, full compliance, and seeking the sample average treatment effect.

How does a Three-arm design function?

It has a Control, Treatment 1, and Treatment 2, where each unit gets exactly one condition.

Define a Factorial Design.

A design where units receive combinations of multiple treatments (e.g., Treatment A, Treatment B, or both).

What are the three things a Factorial Design allows researchers to study?

Effect of A, effect of B, and the interaction between A and B.

In an Encouragement Design, what does the researcher actually assign?

Encouragement (e.g., a reminder), not the treatment uptake itself.

What is the primary inquiry (estimand) in an Encouragement Design?

The Complier Average Causal Effect (CACE).

What is Intention to Treat (ITT) in an encouragement design?

The effect of the assignment/encouragement itself on the outcome.

What is a Sharp RDD?

A design where treatment is perfectly determined by a strict "hair-trigger" cutoff (probability goes from 0 to 1).

What is a Fuzzy RDD?

A design where the cutoff creates a substantial jump in the probability of treatment, but not a perfect one.

What is the Continuity Assumption in RDD?

The assumption that potential outcomes would be smooth/continuous at the cutoff in the absence of treatment.

What does the vertical distance at the cutoff in an RDD represent?

The Local Average Treatment Effect (LATE).

What is the Parallel Trends Assumption in Difference-in-Differences (DiD)?

The assumption that treated and control groups would have moved in the same way over time if treatment hadn't happened.

What is a Stepped-Wedge Design?

A design where treatment is rolled out sequentially over multiple time periods until everyone is treated.

Why is a Stepped-Wedge Design often used for ethical reasons?

It is useful when it would be unethical to withhold treatment from a control group forever.

What is a Randomized Saturation Design used to study?

Spillovers, peer effects, and interference within clusters.

How is treatment varied in a Saturation Design?

Different clusters are assigned different proportions (saturation levels) of treated individuals.

What is Sorting in an RDD context?

When units have an incentive to manipulate their position to be on one side of a threshold (e.g., welfare eligibility).

What is a Density Check in RDD?

A test to see if there is a sudden change in the number of observations on one side of a cutoff, which may indicate sorting.

What is a Balance Check in RDD?

Checking if observable covariates (like age or gender) "jump" at the cutoff; they should remain smooth.

What is a Placebo Cutoff test?

Re-running an RDD at an arbitrary, fake threshold to ensure no effect is found there.

What is the purpose of a Placebo-Controlled Experiment?

To account for the placebo effect (the effect of believing one is being treated).

Define Conjoint Experiment.

An experiment where respondents evaluate profiles with multiple attributes to study multidimensional preferences.

What does AMCE stand for in conjoint analysis?

Average Marginal Component Effect.

What is an Audit Experiment designed to detect?

Discrimination or preferences (e.g., using identical CVs with different names).

What is the core mechanism of a List Experiment?

A sensitive item is hidden in a list of non-sensitive items to reduce response bias.