Research Methods in Political Science: Causality, Design, and Measurement Poli stats 4001 MIDTERM

Science

A strategy to understand the world using (empirical) statements that can be tested to see whether they are wrong.

The Falsification Principle

An explanation is scientific if it provides empirical implications: statements that can be tested and conventionally proved wrong.

Scientific Statement

An empirical statement that is testable and must be potentially refutable.

Non-Scientific Statement (to avoid)

Opinions, tautologies (a statement that is true by definition), and unobservable statements.

Political Science

The scientific study of political phenomena.

What Political Science is NOT

Preferences (opinions about politics), normative theories (statements about how the world should be), or trivia (facts about politics).

Goal of Political Science

To develop and empirically test causal theories about politics.

Causal Theories in Polisci

They are probabilistic (e.g., more likely), not deterministic (i.e., not certain to occur every time).

Independent Variable (X)

The variable that represents the causal factor; the 'cause'.

Dependent Variable (Y)

The variable that represents the effect; the 'outcome' or phenomenon of interest.

Causal Theory

A tentative conjecture about the causes of some phenomenon of interest. It is a claim that X→Y.

Causal Hurdle 1

Is there a credible mechanism that connects X to Y? (The theoretical explanation/why X→Y).

Causal Hurdle 2

Can we eliminate the possibility that Y causes X? (Ruling out reverse causation).

Causal Hurdle 3

Is there a covariation between X and Y? (Is there a correlation/relationship?).

Causal Hurdle 4

Have we controlled for all confounding variables Z that might make the association between X and Y spurious?

Confounder (Z)

A variable Z that is associated with both the independent variable (X) and the dependent variable (Y).

Spurious Relationship

An association between X and Y that is not causal, but is instead due to an uncontrolled confounding variable Z.

Research Design

A strategy to collect and analyze data that allows us to rigorously answer causal hurdles 2, 3, and 4.

Experiment

A research design where the researcher both controls and randomly assigns values of the independent variable (X) to the subjects.

Random Assignment

Ensures that the control and treatment groups are statistically identical to each other, making the treatment the only systematic difference.

Internal Validity

The degree to which a study produces a sound estimate of a causal effect for the population under study. It is the crucial goal of experiments.

Observational Study

A research design where the researcher measures the values of X and Y as they naturally occur, without controlling or assigning the independent variable.

External Validity

The degree to which the conclusions from an analysis of a given set of data would hold for other cases or in other settings. It is the crucial goal of observational studies.

Dealing with Hurdle 4 in Experiments

Random assignment to treatment and control groups eliminates the problem of confounding variables Z (the expected Z treatment = Z control).

Dealing with Hurdle 4 in Observational Studies

Attempting to control statistically for potential confounding variables Z.

Causal Inference Challenge in Observational Studies

Due to the difficulty of controlling for all possible confounding variables Z, conclusions about causality must be more tentative.

Data

A systematic record of a series of observations.

Variable/Feature

A concept that can take on more than one value, represented by a column in a data set.

Observation/Case

The unit of analysis being studied, represented by a row in a data set.

Data Set

A spreadsheet-like matrix where the rows are the observations and the columns are the variables.

Cross-Sectional Data

Data where a single variable is measured for multiple units (cases) at a single point in time.

Time-Series Data

Data where a single variable is measured for a single unit across multiple points in time.

Time-Series Cross-Sectional (TSCS) / Panel Data

Data where multiple variables are measured for multiple units across multiple points in time.

Unit of Analysis

The entity you are describing or analyzing (e.g., individuals, countries, elections).

Measurement

The process of creating and collecting data for a variable.

Operationalization

The process of moving from an abstract concept to a concrete measure that can be empirically evaluated.

Reliability

A measure that is repeatable and consistent; applying the same measurement rules to the same case or observation will produce identical results.

Validity

A measure that represents the concept it is supposed to measure, whereas an invalid measure measures something other than what was originally intended.

Measurement Error

The difference between the true value of a concept and the measured value.

Systematic Error (Measurement Bias)

A consistent pattern of over- or under-reporting that pushes the measured value consistently in the same direction (e.g., social desirability bias).

Random Error

An error that affects the measurement of a variable in a completely arbitrary way.

Face Validity

Does the measure appear, on the surface, to be measuring what it claims to measure? (The 'eyeball' test).

Content Validity

The degree to which a measure includes all the essential elements of the concept being measured.

Construct Validity

The degree to which the measure is related to other measures in a way that is consistent with the theoretically expected relationships.