Lecture 1: Theory, Measurement, and Design

• verifiable and repeatable

• falsification

• probabilistic claims

benefits of observation

observation

individuals with more years of formal education turn out to vote more often than those with less education

theory

education increases one’s propensity to turn out to vote

political science

study of politics based on theory and observation

inference

best “guess” about an unknown given known information (uncertain by defintion)

causal inference

unknown causal relationship between two or more variablesd

descriptive inference

unknown fact about a single variable

• theory that expects X to affect Y

• evidence of correlation between X and Y

valid causal inference requires…

correlation

the values of two variables tend to move together

prediction

knowing the value of X helps us predict the future value of Y

causation

change in the value of one concept tends to produce changge in the value of another concept

deterministic causal relation

cause (X) is always present when outcome (Y) occurs

probabilistic causal relation

cause (X) usually present when outcome (Y) occurs, outcome occurs with some likelihood when cause is present

profile of interest

a specific set of characteristics or variables used to identify or analyze a particular group of subjects in research (unit of interest, variation in some characteristics of a population, quantity and qualitative judgements)

hypotheses

testable causal relationships among concepts

falsifiable hypotheses

hypotheses that can be proven false through observation or experimentation.

theory

causal explanations about the cause-and-effect relationships among interrelated concepts; simplified “snapshot” of reality

dependent variable (Y)

the effect or outcome being caused

independent variable (X)

the suspected cause

causal relationship

how and why change in the value of one concept influence the values of another conceptwith a focus on identifying the direction and strength of influence.

deduction

theory first, then empirical testing

induction

observation first, then theory construction to explain what was observed

hypothesis

an expectation about co-variation between the values of two (or more) concepts

positive relationship

the values of x and y move in the same directionn

negative relationship

the values of x and y move in opposite directions

null hypothesis

the expectation of no relationship between two concepts (x and y varies randomly or does not change value)

statistical hypothesis testing

collecting data to assess evidence against the null hypothesis

puzzle

we expect two cases to have similar outcome, but they have different outcomes

spatial variation

different units measured at same time

temporal variation

same unit measured at different times

credible casual mechanism

An explanation that connects cause and effect, supporting a causal inference by outlining how a treatment leads to an outcome.

endogeneity/reverse causation

A situation where the predictor variable is correlated with the error term, leading to biased estimates due to possible feedback loops between cause and effect. (could y cause x?)

co-variation

The degree to which two variables change together, indicating a relationship between them.

spurious correlation

A false relationship between two variables, caused by a third variable that influences both. This can lead to misleading conclusions about causation.

operational defintion

a set of instruction that describe how to measure the value of your concept in the empirical world

unit of interest

The primary object or entity that a study focuses on for measurement and analysis (individual, states, countries, etc.)

variation of interest

The specific changes or differences in a variable that researchers seek to understand or measure within a study (over time, between units, etc.)

validity

the extent to which a concept or measurement accurately reflects the intended variable or construct.

reliability

The degree to which a measurement produces stable and consistent results over repeated trials. Reliability is essential for ensuring the accuracy and dependability of data in research.

systematic error

tendency to assign values that are either too high or low (bias)

random error

equal likelihood of assigning too high and too low values

face validity

The extent to which a test appears to measure what it is supposed to measure, based on subjective judgment.

content validity

extent to which a test measures the intended content or construct.

construct validity

The degree to which a test accurately measures the theoretical concept or construct it claims to assess, often evaluated through various forms of evidence and theory.

reliability

extent to which re-application of a measurement method produces identical values for a variable

test-retest reliability

same measurement to observations at different points in timeal

alternative form reliability

two different measurers of same concept at two different times

split-halves reliability

split cases and use two different measures of same concept at same time

inter-rater reliability

multiple coders of same case

bias

systematic measurement error, measurement is reliable but is consistently low or high

discrete variables

Variables that take on distinct, separate values without intermediate values.

continuous variables

Variables that can take on any value within a given range, allowing for infinite values between them.

nominal level

categories only — cannot rank themor

ordinal level

values can be ranked, but distance between categories unknown

interval-ratio level

numbers — the distance between values has the same value across all values

design

procedures we use to draw inferences (test hypotheses)

• history (concurrent events)

• maturation (passage of time)

• selection (choosing y)

threats to internal validity

maximize comparability

the units should be identical except for the independent variable of interest

random selection

each case has the same chance of being in the experimentr

random assignment

each case has the same change of being assigned to control or treatmentgroup, thus ensuring that the groups are comparable at the start of the experiment.

randomization

makes the two group identical (on average) in all ways except for the treatment, assuming large enough groups

• the value of many independent variables cannot be randomly assigned

• differences between lab and actual world (external validity)

• convenience samples

limits of experimental design

field experiment

randomly assign individuals into groups, but perform the manipulation in the real world

natural experiment

an event outside the social scientist’s control separate people into “control” and “treatment” groups

observational study

take world as it is and study naturally occurring differences across units or over time

cross-sectional

different units measured at same time; variation is across units

time-series study

same unit measured at different times; variation is over time