Research Methods Midterm

What is the empirical approach

• using quantitative methods to research political science. It assumes that there is a lot of uncertainty and it does not exclude qualitative data per se, since it is not as exact a science as the hard sciences. But the empirical approach uses science that is the most reputed in political science

• information is made public so that other people can critique, replicate, and it adds to the social discourse

• easily replicable

• inference shines through based on empirical information

capable of being verified or disproved by observation or experiment

problems with documents

bias: might not paint full picture, could be a distorted account

selectivity: what is available might not representative of all accounts

Distinguish empirical and normative claims

An empirical claim asserts something about observable reality—it can, in principle, be tested, confirmed, or refuted through evidence.

A normative claim expresses a value judgment about what is good, bad, desirable, or just—it cannot be settled by data alone because it depends on ethical, moral, or ideological commitments.

Explain what it means to “make inferences”

To make conclusions on content that is beyond what the data can prove, to make inferences about the politics or society

The very abstract, even unrealistic nature of statistical models makes the rules of inference shine through

Describe the major assumptions, strengths, and limitations of the empirical approach

• assume that social constructs and institutions are reality, there is value to quantifying political science and it can keep track of patterns

• -helps to have a process of coming up with an idea to examine, make a hypothesis, make sure you are defining things properly, figure out how you will collect data, what can be quantified, and see if hypothesis is supported

• Limitations are that there are aspects like values that are difficult to quantify

what are the 4 characteristics of scientific research?

1. the goal is inference

2. procedures are public

3. conclusions are uncertain

4. the content is the method

What are the differences between nonempirical and empirical research?

Nonempirical

• Goals: to understand behavior to interpret actions

• Assumptions: social facts are “constructed;” institutions are social creations observation is not generally possible because our very senses are affected by culturally defined and imposed prior beliefs, totally value-free research is impossible

• qualitative

• methods: qualitative analysis (ethnography, content and document analysis, study of discourse)

empirical

• causal explanations and predictions of individual and institutional behaviors, general theory and laws, information of practical use, value-free knowledge

• 
  

What does the Johnson et al reading say

Describe the characteristics of good research questions

• are researchable

• are interesting

• move beyond WHAT to the WHY and HOW

• often center on puzzling variation

Name some common ways why research questions would be considered bad?

weaker questions might be too narrow, broad, predictive, or difficult to answer empirically

research questions might be not researchable, uninteresting, or don’t move past the “what” question to the “why” and “how”

Why is this a bad question: Will the US still exist in 500 years?

Not researchable b/c we cannot answer the question using existing data or methods. it is speculative and unbounded in time

How would you fix the question, will the US still exist in 500 years?

reframe to something empirically observable: what factors explain the longevity of constitutions in democratic states?

Why is the question a bad question: why does my hometown mayor support public parks

it is uninteresting: the answer only matters to a handful of people, does not connect to broader debates in political science

How do you fix the question Why does my hometown mayor support public parks?

broaden the scope: reframe to something empirically observable: why do some mayors prioritize public spending on parks while others cut it?

Why is this a bad question: How many women are there in the Colombian legislature?

Descriptive “what” question

• answerable with single statistic

• better to ask why as well: why did women’s representation in the Colombian legislature increase following the adoption of gender quota laws?

why is this a bad question: why do authoritarian regimes use repression?

no real variation, no real puzzle

points to something obvious, not puzzling

fix: why do some authoritarian regimes rely primarily on cooptation, while others rely primarily on violent repression? - regimes with same type of government diverge in their rools of control is puzzling

Explain the importance of variation in empirical research

• Variation is the differences in the thing we want to explain

• Political outcomes often vary across people, places, across time, across otherwise similar cases gives us something to explain

• Variation gives us something to explain

What does the Firebaugh 2008 reading say

What does the King, Keohane, and Verba 1994 reading say

Explain what it means for a study to make a scholarly contribution

it answers the question, “so what” why does answering the question matter”

a question deserves to be studied when it contirnbutes to what sholars already know

a contribution explains what we learn from a study that we did not know before

Describe common ways a particular project can contribute to existing knowledge

• build on what others have found

• challenge existing findigns or explanations

• ask questions that existing scholarship has overlooked

• bring new evidence to unresolved questions

• disappearance test: if some article had never been written, what would political science people not know had it not been published

Identify the central contribution of a published political science article (Pager)

starting point: we know race and incarceration separately affect employment. We know less about how they interact

EXTENDS: combines the considerations of race and a felony in job applications

stakes: shows the consequences of incarceration cannot be understood separately from race, helping to explain reproduction of racial inequality

How can you evaluate whether a proposed research question has the potential to make a meaningful contribution

Round 1: what does your article contribute?

Disappearance test: if some article had never been written, what would political science people not know had it not been published

Round 2: compare contributions

stakes/so what — why does knowing matter?

what does fearon and laitin say 2003

starts with ethnic divisions are thought to drive civil war

as an intervention, it CHALLENGES: redirect explanation to conditions that favor insurgency

stakes: changes how we explain civil war — away from ethnic division and toward conditions that make insurgency feasible

what does geddes say 1999

the starting point is that existing democratization findings don’t generalize

it CLARIFIES in order to distinguish types of authoritarian regime

stakes: helps explain otherwise inconsistent findings about how and why authoritarian regimes break down

Explain the purpose of a literature review in political science research (3)

• job 1: it establishes the contribution

  • argumentative, establishes a claim and support it by synthesizing important literature

  • establishes what we know

  • establishes what remains

  • establishes your contribution

• job 2: Establishing what remains

  • what scholarship has and has not established

  • critically assess evaluative strengths, limitations, implications of existing scholarship

• job 3: how you can contribute

  • explain how your proposed research will contribute to existing scholarship

  • LR should make the need for it clear

Differentiate between a literature review and a simple summary of readings

• a lit review should synthesize important literature

• they have a proper structure of grouping existing scholarship

• assess what a body of scholarship accomplishes and what is left unresolved

• use citations

• argue for the author’s case for future research

• ex “Scholars generally agree that ___ but offer several explanations for why it occurs”

recognize common structures of lit reviews

some ways you can organize a lit review is

1. what scholars generally agree on

2. competing explanations for why

3. an important limitation of existing evidence

4. what remains unsolved

What does Howard 2017 say

What does Feldman and luna 2022 say

*Distinguish between a concept and a measure (?)

Concept:

• do not stipulate definitions

• think about real cases that exemplify the concept and build your definition in reference to cases

• aim for a definition that correctly covers as many cases as possible

Measure:

• indirect measures of a concept

• we must think about visible manifestations of the concept and instead measure them

• we cannot measure democracy directly, democracy is an abstraction and therefore invisible

*Develop a clear conceptual definition

entails precise articulation of meaning

• to define a concept, think about real cases that exemplify the concept, build your definition in reference to the cases

• aim for a definition that correctly covers as many cases as possible

• if one or two cases “escape” your definitino, that is okay

• ex: In class we talked about what is democracy

  some possibilities were competitive elections, full suffrage, political righs and freedoms, majoritarianism, protection of minorities

  scholars don’t agree on the definition of democracy

Explain validity and evaluate whether a measure captures the intended concept

• A valid measure accurately represents the underlying concept

• A reliable measure repeatable and consistent

  • applying the same measurement to the same case or observation will produce identical results

What makes for a good measure?

Accuracy = clarity + validity + reliability

• A valid measure accurately represents the underlying concept

• A reliable measure is repeatable and consistent

  • applying the same measure to the same case or observation will produce identical results

what does it mean when a measure is accurately representing the underlying concept

validity

What is it called when a measure is repeatable and consistent?

reliability

What makes a measure accurate?

clear, valid, reliable

*What is face validity?

*What is content validity?

*What is construct validity?

What does Kellstedt and Whitten 2009 say

What does Gladwell 2011 say

What are the 4 levels of precision aka measurement scales?

1. Categorical

   1. Nominal

   2. Ordinal

2. Numerical

   1. Interval

   2. Ratio

Explain the nominal measurement scale

• variable where values denote unordered categories

• ex: voter turnout 1 - voted, 0 - did not vote

• race 1 - white 2 hispanic 3 asian 4 native american

• electoral system proportional, first past the post, mixed member proportional

What is the measurement scale that denotes unordered categories?

nominal measurement scale

What is the ordinal measurement scale?

Variable where values denote ordered categories

ex: political ideology 1 -very liberal 2 liberal 3 modest 4 conservative 5 very conservative

presidential approval (highly disapprove-highly approve), perceived corruption (very corrupt - very clean)

What measurement scale denotes ordered categories?

ordinal

what is the interval measurement scale?

variable where the differences between values are meaningful and consistent

ex: political knowledge score on a test (0-100)

put yourself on a scale of 0-10 how conservative are you, thermometer politics

What measurement scale denotes differences between values that are meaningful and consistent

interval

what is the ratio measurement scale

variable where the differences between values are meaningful and consistent, and 0 denotes the absence of the variable

ex: consecutive years of democracy, count variables: diplomatic incidents, militarized conflicts, homocides

anything that can be counted

variable where the differences between values are meaningful and consistent, and 0 denotes the absence of the variable

ratio measurement scale

Explain what additional information we gain as measurement becomes more precise

a measure is more precise when it allows us to make finer distinctions between observations

T/F: a more precise measurement means it is a more accurate measurement

false

what is the difference between categorical and numerical variables

nominal and ordinal variables are both categorical variables

interval and ratio are both numerical

both nominal and ordinal variables are…

categorical variables

both interval and ratio variables are …

numerical variables

What is random error?

When a measure’s value is incorrect for a given case or set of cases

ex: country is assigned 98 when correct value was 96

it is relatively harmless

the measure lacks precision, but represents the underlying concept on average

Identify and explain measurement bias

bias is not random but systematic error

• ex: asking “do you support the government wasting taxpayer money on welfare programs for people who refuse to work?”

• more problematic than random error because it undermines the accuracy of our measure. it no longer represents the underlying concept

Evaluate the tradeoffs involved in different ways of measuring a concept

Apply these criteria to the measurement of democracy and other political concepts

What does Johnson et al 2020 say

*What does Munck and Verkuilen 2002 say; how do they propose evaluating conceptions of democracy?; what problems do munck and verkuilen identify in translating concepts into observable measures?; according to the authors, what would a researcher need to do to construct a defensible measure of democracy?

What is a theory

• general explanation of a phenomenon or relationship

• explains how or why something happened

• generates multiple testable expectations

• ex: states with greater administrative capacity are better able to implement public policies effectively

what is a hypothesis

specific, testable statement about an expected relationship

can be evaluated using evidence

ex: higher capacity states → faster vaccination rollout, coverage, more effective tax collection

Distinguish between descriptive and causal hypotheses

• descriptive hypothesis: two variables are related

• ex: countries led by women had lower COVID 19 mortality

• causal hypothesis

  • changes in one variable produce change in another

  • women leaders caused countries to experience lower COVID 19 mortality

Recognize correlated, partly spurious, and completely spurious relationships

completely spurious C causes A (iv) and B (dv)

partly spurious C connects to A and B, but there is a dotted line between A and B

correlated just means they move in the same direction

• can have correlation while it being spurious

• ex: increased ice cream consumption and increased shark attacks correlate

because of this, researchers should identify rival explanations for a relationship and evaluate teh evidence

explain how researchers can investigate and rule out potential sources of spuriousness

discover the causal mechanism, test for confounding variables

Articulate the meaning of causality and the criteria for establishing a causal relationship

know the direction one IV directly causes DV to change, helps to have CM

Distinguish deterministic from probabilistic relationships and recognize each in examples

probabilistic increases the probability that something will occur while deterministic is like if they have this then it causes the dv to be like this (yes/no)

Explain why establishing causality matters, using environmental justice research as an example

identify real patterns

correlation does not equal causation

establishing relationship is not the same as establishing causation

• ex: low income communities and communities of color are disproportionately exposed to pollution — want to figure out why

What did Howard 2019 say

What did Kellstedt and Whitteen 2013 say

What did Banzhaf, Ma and Timmons 2019 say

How do you move from observing a relationship to making a credible causal claim?

challenge 1: causal direction

• 
  

challenge 2: causal mechanism

• explain how causation works

• ex: siting firms locate polluting facilities in already disadvantaged communities

What is the causal direction of 2 explanations for correlation btw ses & race with pollution exposure?

siting (polluting facilities placed in already disadvantaged communities) SES & race → pollution exposure

sorting (pollution affects housing markets and residential decisions, changing who lives near polluting facilities) pollution exposure → SES & race

What are the possible mechanisms for siting facilities in disadvantaged communities?

• incentives: lower land costs or proximity to labor and transportation

• political power: firms anticipate less organized opposition or lower costs of securing approval

• regulatory decisions: government permitting and enforcement may contribute to unequal exposure

what are some possible evidence that firms anticipate less organized opposition or lower costs of securing approval?

• firms consider anticipated community opposition when evaluating potential sites

• communities with less capacity to organize are more likely to be selected, all else equal

• differences in political mobilization are linked to differences in siting outcomes

Define and distinguish causal mechanisms and effect modifiers

causal mechanism is how the IV impacts the DV

effect modifiers change how the IV impacts the DV??

Identify IVs, DVs, potential sources of spuriousness, mechanisms, and effect modifiers in arrow diagrams

independent variable X

dependent variable Y

source of spuriousness Z

intervening variables:

mechanism M

effect modifier W

Construct and use arrow diagrams to represent and critically evaluate arguments

step 1: relationship

• sketch the core relationship between IV and DV

step 2: spuriousness

• add at least one potential spuriousness identified by authors

step 3: mechanisms

• add at least one mechanism through which the IV influences the DV

what is the relationship, source of spuriousness, and mechanisms between social media use and anti-refugee attacks

IV social media use

DV anti-refugee attacks

Z sources of spuriousness: economic conditions, demographics, far-right political support, media consumption

M mechanisms: exposure and amplification, attitude reinforcement, perceived social norms

What does Howard 2017 say

What does Taub & Fisher say 2018

Identify and diagram causal hypotheses and their proposed mechanisms in published scholarship

• an arrow from X to Y denotes a causal relationship, not necessarily deterministic

what does complete spuriousness look like in a diagram?

line from X to Z and line from Z to Y with no line between X and Y means relationship between X and Y is fully explained by the third variable Z

how do you diagram partial spuriousness?

the observed relationship between X and Y is partially explained by the third variable Z. X and Y remain related, but the relationship may be weaker than first appeared

ex: Ross article natural resource wealth and democracy

what does a diagram explaining multiple causes X and Z look like?

X and Z are distinct causes

the relationship between X and Y is not explained by Z

what does a diagram with an effect modifier W look like?

dashed line connects W to the arrow between X and Y

the dashed line is an annotation, not a causal arrow

*Explain how scholars evaluate causal relationships by accounting for potential sources of spuriousness

CM?

*Distinguish evidence of a relationship from evidence supporting a particular causal mechanism

What are 2 additional challenges with establishing causation?

1. causation is often probabilistic

   1. a cause might change the probability of an outcome without invariably producing it

   2. Hdeterministic vs Hprobabilistic

2. causation may change in different contexts

   1. Ex: X criminal record Y employment Z race — the employment penalty of a criminal record differs by race, Z becomes an effect modifier

what is the trouble with the hypothesis: a criminal record reduces the employment opportunities

the cause is not the same effect in every context

race is an effect modifier, which changes the magnitude or direction of the effect of X on Y

what is an effect modifier

changes the magnitude or direction of the effect of IV on DV

what is a deterministic hypothesis compared to a probabilistic hypothesis no political influence and being selected for a polluting facility

H deterministic: greater political influence prevents a community from being selected for a polluting facility

H probabilistic: greater political influence reduces the probability that a community is selected, all else equal

What is the causal diagram and all the parts of reading Does Oil Hinder Democracy

Oil and democracy is the core relationship

the causal mechanisms are rentierism, repression, and modernization

rival explanations are wealth, islam, location, country history/culture, common temporal shocks

what does Ross 2001 Does Oil Hinder Democracy find

more oil wealth → less democracy (negative relationship)

analyzing the data produces evidence of covariation. by itself, this is not evidence that oil wealth causes less democracy

taking muslim population share into account does not eliminate relationship betwn oil and democracy, rival explanation is less plausible

states that rely more heavily on taxation are more likely to be democratic (taxes are related to regime, not oil)

how do you prove causal direction in oil hindering democracy case study

greater oil wealth should precede lower levels of democracy

ross measures oil five years before the corresponding measure of regime, asks if past oil wealth is associated with subsequent democracy

relationship is still negative — greater past oil wealth is associated with less democracy 5 years later

*wtf is covariation

what is rentierism?

if oil reduces democracy by suppressing demands for democracy, we expect greater oil wealth to reduce taxation and/or increase government spending

While many states export resources or license their development by foreign parties, rentier states are characterized by the relative absence of revenue from domestic taxation, as their naturally occurring wealth precludes the need to extract income from their citizenry and is a challenge to developing civil society and democracy

what is a research design

a strategy or blueprint for answering a research question