Political Science- Research Designs and Methods

Types of Inference

Descriptive-

Causal-

Research questions…

seek to explain a phenomena or pattern. A good research question will have

• Causal relationship/explanation

• Variables

• Not be subjective/normative

• Not have an empirical answer

Science is…

• Empirical

• Repeatable

• Not definitive

• Falsifiable (able to be proven false)

• Generalizable

• Explanatory

Steps to the scientific method

1. Ask a research question

2. Develop a theory (a reasonable explanation to a phenomena)

3. Develop a hypothesis (a directional, testable statement of inference between variables)

4. Experiment

5. Fail to reject or reject null

6. Revise theory if applicable

The null hypothesis tests whether…

something is NOT occurring, there is no effect between variables. This is because it is impossible to prove that something is true.

Social science deals with _ relationships, not _ ones because…

1. Probabilistic (the probability that something is likely to occur)

2. Deterministic

3. …hard sciences deal with deterministic relationships. That is, proving one thing definitely triggers the other.

A causal explanation/mechanism…

describes what causes variation between variables. It is the “why” something is going on.

What is an identification strategy?

The way a researcher uses observational data (not randomized) to approximate an experiment. It is an approach you use to describe your casual process when you can’t do a real experiment.

What is mode of inference?

The mode of inference is how you study your population via

• Sampling

• Which population

• Assumptions

What are the three main problems of science?

1. Induction: We can’t monitor all human behavior, so we rely on samples, but samples may be biased.

2. Free-will: Hard to generalize human behavior to a law-like reductionist statement.

3. Oversimplification: Even if we have findings they are not applicable to everyone because of human complexity. Models ignore important variation in the world.

What is a theory?

A theory is a reasonable explanation to a phenomena/research question that includes why its proposed answer is correct. Theories tell causal stories.

• X causes Y because….

Theories are made of four parts:

• Variables/concepts

• Variation between variables (a change in one causes a change in another)

• Causal mechanisms, why something happens

• Suggest empirical implications (aka hypothesis)

Part one of a theory…

Identify key concepts and variables

• X, independent variable

• Y, dependent variable

• Unit of Analysis: Who/what you’re describing

Example: More time on social media leads to increased aggression.

• Time on social media: X

• Aggression levels: Y

• Unit of analysis: Adults 18-40

Variation between variables happens over…

• Time (Longitudinal)

• Space (Cross-sectional)

• Both

What is a model? What is a downside to a model?

A simplification and visual representation of your theory. Models are oversimplifications and do not account for real-world variation.

Positive relationships…

Move in the same direction

• Example: +X → +Y

Negative relationships…

Move in opposite directions

• Example: -X → +Y

Types of variables within models

• Intervening/Mediating: X→ X.1→Y

• Control Variable: X1 (Class Size)vs X2 (Teaching experience); X1 → Y (Test Scores)

• Antecedent Variable: W→ X→ Y

• Moderating Variable: Changes the effect of X; X → Y

  • X = Class Size; Y= Test Scores, Moderating = Instruction time (similar to intervening, but not caused by X)

General themes of hypothesis are…

1. General (can apply to as any units as possible)

2. Precise (have clear variables) /Imply levels of analysis

3. Falsifiable (H0)

4. Empirical (observable and measurable)

What is the difference between a hypothesis and a theory?

A theory gives a rationale behind the hypothesis which is testable and directional (or observable and measurable). Theories suggest hypotheses. Theories will always contain a causal mechanism.

Reading: Gerber, Green, and Larimer

What are the main points of this paper?

Gerber et al. study social pressure and voter turnout. They conduct an experiment where they send mail ballots and find that voters are more likely to turnout to vote if are publicized to do so, demonstrating the importance of social pressure on voter turnout.

They had two hypotheses:

H1: citizens are more likely to vote as their intrinsic rewards

increases.

H2: citizens are more likely to vote with increases in the

perceived probability that his or her participation is known

to others

Reading: Ziblatt

What are the main points of this paper?

Ziblatt explores why there is electoral fraud in new democracies. His findings suggest inequality in landholding, wealth and power (social inequality) contribute to fraud. He conducts his investigation by analyzing data from German parliament in the 19th and 20th centuries.

H1: “in electoral districts with higher levels of landholding

inequality, the incidence of electoral fraud will be greater”

X: Social inequality

Y: Fairness of election

What are the two different types of evidence?

Qualitative and Quantitative Data

Quantitative research…

deals with numerical data, usually large N studies with random sampling, and focuses on variables.

Descriptive

Qualitative research…

deals with non-numerical data (like texts), usually small N or purposive sampling, and focuses on causes.

Causal/Inferential

Induction deals with…

looking at the literature; from observations to patterns to conclusion

Deduction means…

testing; the steps of the scientific method; theory →H→test..etc.

Reading: Gade

What are the main points of this article?

Gade researches social isolation and resistance in Palestine/Israel. She suggests checkpoints encourage resistance through community ties.

What are some advantages to qualitative research?

Qualitative research allows researchers to better understand causal mechanisms, specific cases and groups.

What is triangulation?

It is a method of converging quantitative and qualitative evidence.

Quantitative and qualitative methods both use _ and _ data.

Primary and Secondary

Primary Data is when…

a researcher collects their own data

Secondary Data is when…

a researcher uses existing data

What are some forms of quantitative data?

Micro data: Raw data

Aggregate: Grouped

Multilevel: Combine different units of analysis

Descriptive statistics deals with…

Central tendency and spread (SD/E)

Statistical inference…

is the process of making probabilistic statements (from sampling distribution) about a population from a sample

What are some pros and cons of random sampling?

Pro: Most likely method to ensure fair representation

Con: Expensive, not every random sample is reliable

Statistical significance is when…

the p-value is less than 0.05%. This is because in the case that the null is true, there is no relationship between the variables, the likelihood of having no relation is small which gives support to our hypothesis. If its greater than 0.05% then the likelihood that there is no relationship between the variables is larger so it does not support our hypothesis.

What is the difference between standard deviation and standard error?

Standard deviation describes the variation within a sample; how much a score differs from the sample mean. Whereas, standard error describes variation of your sample across different samples. It is “how representative your sample is of the larger population”. You can reduce error by increasing sample size (central limit theorem).

Standard Error tells us…

how likely we are to observe our sample statistic given the population parameter.

Confidence Intervals

intervals that include the parameter 90-95% of the time

Point Estimate ± tcrit ∗ st.error

Margin of Errors

are the degrees of uncertainty around an estimate based on hypothetical sampling at x%

tcrit ∗ st.error

What are the three types of bias when it comes to causality?

1. Endoengeity

2. Common Cause/ Confounding Variable

3. Sampling/ Selection bias

Causal inference

requires counterfactual conditions

Exogenous means…

• the x does not depend on y

• nothing causes x (no influencing external factors)

Randomization and control makes a treatment…

exogenous

Observational designs rely on…

observing trends in data and arguing for causality

In Quasi/Natural experiments

the researcher does not have control over the x, but the assignment is as-if natural and they compare to a control group

Internal validity…

establishes causal inference. A change in x causes a change in y

External validity…

how generalizable a study is; the extent to which it matches what happens in the real world

Why are experimental designs so good at establishing causality?

• control administration of treatment

• randomization

Reading:

• What does Turner discuss in their paper?

Turner compares the bias ratings to CNN and Fox news in a lab experiment

Hypotheses: On average, viewers will rate news reports identified as being from CNN as liberally biased and news identified as being from Fox as conservatively biased regardless of the news’ actual source.

This was a lab experiment.

What do Bond et al examine?

How does online social interactions influence impact voter mobilization?

They found that social media has a huge impact on voting behavior.

This was a field experiment

What are threats to internal validity?

• Endogeneity

• Confounding (or omitted factors)

• Selection Bias

Internal Validity- Experiments

Strengths

• Confounding Factors: Random Assignment

• Endogeneity: Administer control of treatment

External Validity- Experiments

Strengths

• Can be replicated

• Falsifiable

Weakness

• Artificial setting

• Oversimplification

• Sampling issues

Internal Validity- Quasi/Natural

Strengths

• Endogenity: Using established causal evidence to argue for (exogenous) causality

• Confounding: As-if random assignment

Weakness

• Cannot ensure exogeneity

External Validity- Quasi/Natural

Strengths

• Pretty strong because it happens in real-world

Weakness

• Consider: Do the participants represent the broader population you want to generalize to?

Internal Validity- Observational

Strengths

• Endoegnity: Temporal order

• Confounders: Careful selection/examination of progress, combine multiple sources of information

Weakness

• Generally lower because it is not definitive

External Validity- Observational

Strengths

• Generally high because it relies on established research in the real world

Weakness

• With purposive sampling its hard to say a small subset’s outcome applies to an entire population’s.