Research Design for Political Science Exam 1

Why do scientists favor the null hypothesis?

Scientists favor the null hypothesis as the default position because they want to be cautious about claiming that a relationship exists. The null hypothesis generally says that there is no relationship between the independent and dependent variables. Researchers then ask whether the evidence is strong enough to reject that position in favor of their theoretical expectation.

This reflects the broader scientific principle of skepticism: claims about causal relationships have to survive attempts to disprove them.

Main idea to remember: Assume no relationship until the evidence gives us sufficient reason to reject that assumption.

Briefly explain an independent variable, dependent variable, and theory.

An independent variable (X) is the factor that a theory proposes is a cause or explanation. A dependent variable (Y) is the outcome that we are trying to explain and that is expected to change in response to X.

A theory is more than simply saying that X and Y are associated. It provides a causal explanation for why X should affect Y, usually by identifying a mechanism connecting them.

The important point is that a theory tells us why and how the relationship should occur, rather than simply observing that two variables move together.

Modernization theory says economic development leads to democratic development. Are there potential biases in this causal proposition?

Yes. One important problem is that the proposition already assumes a particular causal direction: economic development → democracy. But the relationship might be more complicated.

For example, reverse causality is possible: democratic institutions might themselves encourage economic development. There could also be a confounding variable that influences both development and democracy, creating an apparent relationship between them even if development itself is not the true cause.

The theory can also reflect selection or historical biases if it was developed largely from observing countries whose experiences seemed to fit that progression.

Observing that richer countries tend to be more democratic does not automatically prove that becoming richer causes democratization.

Why do the authors recommend pursuing both generality and parsimony?

Generality means developing theories that explain more than one extremely specific event or case. A useful theory should potentially apply across multiple situations.

Parsimony means explaining political phenomena with as few unnecessary assumptions or variables as possible. A theory that requires dozens of special conditions may become so complicated that it loses explanatory usefulness.

The goal is therefore to construct theories that are simple enough to be understandable and testable but general enough to explain a meaningful range of political phenomena.

How did the rise of behavioralism change comparative politics after World War II?

Behavioralism pushed comparative politics toward a more explicitly scientific study of political behavior. Instead of concentrating primarily on constitutions, formal institutions, and descriptive accounts of particular countries, behavioralists wanted to identify broader patterns that could be studied systematically.

This meant greater attention to things such as individual behavior, attitudes, political participation, voting, groups, and other observable political activity.

The broader change was therefore from a field dominated by formal institutions and descriptive country studies toward one seeking generalizable explanations of political behavior through systematic empirical research.

How did comparative politics change during the "second scientific revolution" after the late 1980s?

The second scientific revolution pushed comparative politics even further toward theory-driven and methodologically rigorous causal research.

Three changes are particularly important:

Focus: Researchers increasingly moved away from broad descriptions of entire political systems and toward more precisely defined political outcomes and causal questions.

Theory: Approaches such as rational choice and other explicit theoretical models became more influential. Researchers increasingly specified mechanisms and hypotheses rather than simply describing patterns.

Empirical methods: Comparative politics became much more sophisticated methodologically, with increased use of statistics, formal modeling, experiments, quasi-experiments, and improved qualitative methods.

Explain the tension between internal and external validity in single-country research

Internal validity asks whether a study has convincingly identified the causal relationship within the cases actually being studied. External validity asks whether those findings can be generalized beyond those cases.

Historically, this concern about generalizability contributed to the decline of single-country studies.

How do contemporary single-country studies differ from those common in the 1960s and 1970s?

The older single-country study was generally descriptive or theory-generating.

Contemporary single-country studies are much more likely to be theory-testing and quantitative.

older: qualitative/descriptive → theory generation → macro-level politics

newer: quantitative/research-design focused → hypothesis testing → micro/subnational processes.

Why might "know local, think global" be a good strategy for developing theory?

Researchers often understand a particular political phenomenon because they have deep knowledge of a specific case or context. That local knowledge can reveal patterns, puzzles, or mechanisms that someone looking only at broad datasets might miss.

But the researcher should then "think global" by asking whether the explanation might apply beyond that particular case.

So the strategy combines the strengths of specific knowledge with the goal of general theory.

Easy memory: Notice something locally → identify the mechanism → ask whether that mechanism travels.

Mention the three strategies proposed to develop an original theory and explain one.

1. Know local, think global — begin with something you understand deeply and ask whether the explanation can be generalized.

2. Think about how a theory operates at different levels of aggregation — a relationship that exists among individuals might work differently among states, countries, organizations, etc.

3. Take existing theories seriously and look for where they can be extended, modified, or applied differently.

For example, using know local, think global, a researcher might notice that students participate more in organizations when their friends participate. Instead of treating this only as a CMU phenomenon, they might theorize more generally that social networks lower the costs of political participation, which could then be tested in other settings.

"Consider how a theory might work differently at varying levels of aggregation."

The level of aggregation is the level at which we observe political phenomena—for example, individuals, neighborhoods, states, or countries.

A causal relationship found at one level does not necessarily operate identically at another. For instance, a theory explaining why an individual votes may not automatically explain why one state has higher turnout than another.

The authors are therefore encouraging researchers to ask whether changing the unit of analysis changes the mechanism or expected relationship.

Why is this hurdle important: Is there a credible causal mechanism connecting X to Y?

Even if X and Y are strongly associated, we need an explanation for how X actually produces Y.

A causal mechanism identifies the process connecting the proposed cause to the outcome. Without one, a correlation could simply be coincidence or the product of another variable.

Why is this hurdle important: Have we controlled for confounding variables Z that might make X and Y spurious?

A confounder (Z) is a variable that affects both X and Y. If we fail to account for it, we may incorrectly conclude that X causes Y.

Researchers must identify plausible alternative explanations and control for them through their research design or analysis.

Why does a substantial portion of disagreement between scholars boil down to the fourth causal hurdle?

The fourth hurdle concerns whether we have controlled for all relevant confounding variables.

This creates disagreement because researchers can rarely prove with absolute certainty that they have accounted for every possible alternative explanation.

Many scholarly debates about causation ultimately become debates about whether the relationship is genuinely causal or whether some omitted factor generates it.

What is an experiment, and why is it useful in political science?

An experiment is a research design in which researchers manipulate an independent variable or treatment and compare outcomes between groups, ideally with random assignment to treatment and control groups.

Random assignment is especially valuable because, on average, it makes the groups similar except for the treatment. That helps address the confounding-variable problem. If the groups subsequently differ in Y, researchers have stronger grounds for attributing that difference to X.

Experiments are therefore especially powerful for causal inference.

What is an observational study? Difference between cross-sectional and time-series?

In an observational study, researchers observe political phenomena without randomly assigning subjects to treatments. Much political science research is observational because researchers cannot randomly assign things like wars, economic crises, regime types, or constitutions.

A cross-sectional study examines variation across units at approximately one point in time. For example, comparing levels of democracy across 100 countries in 2025.

A time-series study examines variation in a variable over time, usually for the same unit—for example, tracking U.S. presidential approval every month for 30 years.

So:

Cross-sectional = across units.

Time-series = across time.

Explain two drawbacks to experimental research designs.

One drawback is external validity. An experiment may provide strong evidence about causation within the experimental setting, but participants or conditions may differ from real political environments. We therefore have to ask whether the results generalize.

A second drawback is feasibility and ethics. Many politically important variables simply cannot be manipulated. Researchers cannot randomly assign people to dictatorships, wars, poverty, discrimination, or economic recessions merely to observe the consequences.

Not every politically important question can realistically or ethically be studied experimentally.

Explain stratified, purposive, snowball, and quota sampling.

Stratified sampling: Divide the population into relevant subgroups, or strata, and select participants from each. This ensures that important categories are represented.

Purposive sampling: Researchers deliberately select people because they possess particular characteristics, experiences, or knowledge relevant to the research question.

Snowball sampling: Existing participants recommend or connect researchers with additional participants. The sample therefore expands through respondents' networks.

Quota sampling: Researchers establish a target number of participants from particular categories and recruit until each quota is filled.

Give an example of a probing, specifying, and indirect question.

Probing question: follows up on something the participant has already said to get greater depth. --> "You said that meeting changed your opinion. Why did it change your opinion?"

Specifying question: asks for more concrete details about an experience or event. --> "You said the meeting became tense. What happened immediately after the disagreement began?"

Indirect question: asks about other people's views or behavior rather than directly asking respondents to reveal their own position. --> "How do people in your department generally feel about this policy?"

Discuss two strengths and two limitations of interviews.

Two major strengths are depth and flexibility. Interviews allow researchers to understand how participants interpret their experiences rather than forcing them into predetermined response categories. Researchers can also follow up, clarify ambiguous answers, and investigate unexpected information.

Two major limitations are time/resources and bias/reliability.

Conducting, transcribing, coding, and analyzing interviews can require substantial time. Responses can also be affected by memory, social desirability, the interviewer, or participants' incentives to present themselves in particular ways.

Why is rapport important, and how can it be achieved?

Rapport helps respondents feel comfortable enough to speak openly and provide detailed answers. Without it, respondents may become guarded, provide short answers, or tell the researcher what they think the researcher wants to hear.

Leech recommends building rapport partly by making the interview feel like a natural conversation rather than an interrogation.

Its purpose is to create enough trust and conversational comfort for respondents to explain their perspectives fully.

What are grand tour questions, example questions, and prompts?

Grand tour questions are broad questions that invite respondents to describe an experience or topic in their own way. They're useful for opening a topic and discovering what the respondent thinks is important.

Example: "Walk me through a typical day in your job."

Example questions ask respondents to give a concrete instance of something they have discussed.

Example: "Can you give me an example of a time when that happened?"

Prompts encourage respondents to continue or elaborate without introducing a completely new question. They can be verbal—"What happened next?" or "Could you say more about that?"—or even nonverbal pauses.

A useful sequence is:

Grand tour → broad story → example → concrete instance → prompts → more depth.

Pros and cons of open-ended questions

The biggest advantage is depth and validity. Respondents can organize answers according to their own frameworks rather than being forced into categories chosen by the researcher. This is especially useful when studying complex or poorly understood phenomena. Elite respondents also tend to prefer explaining why they think what they think rather than being confined to predetermined choices.

The tradeoff is that open-ended interviewing is time-consuming and expensive.

The method should follow the purpose of the research.

Why do the authors praise systematic coding of elite interviews?

Open-ended elite interviews generate huge amounts of nuanced information. Systematic coding converts that material into categories that can be compared and analyzed without completely eliminating the richness of the responses.

Aberbach and Rockman distinguish manifest codes for explicit responses, latent codes for underlying characteristics not explicitly asked about, and global codes for broader judgments about respondents' styles or frameworks.

Most importantly, systematic coding prevents a researcher from allowing a particularly charismatic interviewee or memorable story to dominate their interpretation of the entire study.

Interviewees have no obligation to be objective or truthful. What does Berry suggest doing about this?

Berry's key point is that an interviewee's statement should not automatically be treated as objective fact.

Researchers should therefore triangulate interview evidence by comparing what respondents say with other interviews, documents, records, and other available evidence.

Researchers should also think critically about why the respondent might be giving a particular answer and recognize that an interview may sometimes tell us more about the respondent's perspective than about objective historical reality.

When does Berry say interviewers have reason to probe?

Researchers should probe when an answer is unclear, incomplete, surprising, contradictory, vague, or especially important to the research question.

Probing is also useful when a respondent says something that conflicts with information from another source.

Discuss two reasons for conducting elite interviews.

One reason is corroboration/triangulation. Researchers can compare interviews with documents, memoirs, archival sources, and other interviews. If independent sources support the same account, confidence in the finding increases.

A second particularly important reason for process tracing is to reconstruct political events and causal processes. Key political actors may know what happened behind closed doors—what alternatives were considered, what disagreements existed, and why particular decisions were made. Interviews therefore reveal aspects of political processes that official documents may omit.

Why does Tansey advocate non-probability sampling? Advantages and disadvantages?

For process tracing, the objective is not to create a statistically representative sample of all political actors. The researcher wants information from the people who were actually important to the event or causal process being studied.

Therefore, non-probability sampling allows researchers to deliberately select key political actors with relevant firsthand knowledge.

The advantage is therefore informational relevance: researchers can concentrate limited time on participants who actually possess evidence needed to reconstruct the causal process.

The disadvantage is limited representativeness.

Non-probability → better for identifying key actors and tracing a specific process.

Probability → better when the objective is population-level generalization.

When is it more appropriate to conduct a case study? Compare it to other methods.

According to Yin, case studies are particularly appropriate when the research question asks "how?" or "why?", when the researcher has little or no control over behavioral events, and when the study focuses on a contemporary phenomenon in its real-world context.

The method therefore differs from an experiment, where researchers manipulate variables and control conditions. It also differs from a survey, which is generally better suited to questions such as "who," "what," "where," or "how many" across many cases.

So case studies are not simply what researchers use when they only have one case. They are a deliberate research design for investigating a phenomenon deeply and contextually, particularly when separating the phenomenon from its real-world context would be difficult.

How does Yin define case studies? Explain the twofold definition.

First, a case study investigates a contemporary phenomenon—the "case"—in depth and within its real-world context, especially when the boundaries between the phenomenon and its context aren't clearly separable.

Second, because the phenomenon and context overlap, the researcher has to deal with many potentially relevant variables and multiple sources of evidence. Case study research therefore relies on multiple sources that should converge through triangulation, and data collection and analysis are guided by prior theoretical propositions.

So Yin's definition is both about:

1. What you're studying: a contemporary phenomenon embedded in its real-world context.

2. How you're studying it: using a rigorous design involving multiple sources of evidence, triangulation, and theoretical guidance.

Discuss the four major types of case studies.

Single-case vs. multiple-case

and

Holistic vs. embedded.

That gives us four types:

1. Single-case holistic: One case with one overall unit of analysis. For example, studying one organization as a whole.

2. Single-case embedded: One case but with multiple units of analysis within it. For example, studying one university but separately analyzing students, faculty, and administrators.

3. Multiple-case holistic: Several cases, with each examined as a single overall unit. For example, comparing several universities as whole institutions.

4. Multiple-case embedded: Several cases, each containing multiple subunits of analysis. For example, comparing several universities while examining students, faculty, and administrators within each.

How many cases? → single / multiple

How many units within each? → holistic / embedded

Yin says cases can be critical, extreme, common, revelatory, or longitudinal. Explain three.

Critical case: A case selected because it provides an especially important opportunity to test a theory or theoretical proposition. If a theory strongly predicts what should happen under these conditions, the case can provide an important test of that expectation.

Extreme or unusual case: A case selected because it is rare, exceptional, or significantly different from ordinary cases. Studying an unusual event can reveal mechanisms or phenomena that would be difficult to observe in normal circumstances.

Revelatory case: A case in which the researcher gains access to a phenomenon that was previously inaccessible to social-science investigation. The value comes from the opportunity to observe something researchers previously couldn't study directly.

Hoop tests vs. smoking-gun tests

A hoop test is necessary but not sufficient for confirming a hypothesis. A hypothesis must pass to remain possible, so failing eliminates it, but passing does not prove it and only somewhat weakens rival hypotheses.

A smoking-gun test is sufficient but not necessary: passing strongly confirms the hypothesis and substantially weakens rivals, but failing does not eliminate it.

Thus, hoop tests are useful for eliminating hypotheses, while smoking-gun tests are useful for confirming them.

Straw-in-the-wind vs. doubly decisive tests

A straw-in-the-wind test is neither necessary nor sufficient. Passing makes a hypothesis more plausible and slightly weakens rivals, while failing makes it less plausible, but neither result confirms or eliminates it.

A doubly decisive test is both necessary and sufficient: passing confirms the hypothesis and eliminates rival explanations. Doubly decisive tests are rare, but several pieces of evidence can be combined to achieve the same inferential leverage.

Therefore, straw-in-the-wind is the weakest test, while doubly decisive is the strongest.