Chapter 1 Notes: The Science in Social Science
Introduction
The book Designing Social Inquiry by Gary King, Robert O. Keohane, and Sidney Verba focuses on research design in the social sciences, aiming to produce valid descriptive and causal inferences about social and political life.
Emphasizes practical design choices rather than philosophy or specific data techniques (surveys, fieldwork, statistics).
Applies broadly to political science and to other disciplines (sociology, anthropology, history, economics, psychology) and to non-disciplinary areas (legal evidence, education research, clinical reasoning).
The central claim is that good research can be understood as deriving from a single underlying logic of inference, even though quantitative and qualitative styles differ in technique.
Two Styles of Research, One Logic of Inference
Main goal: connect quantitative and qualitative traditions by applying a unified logic of inference to both.
Differences between quantitative and qualitative research are mainly stylistic and methodological; the underlying logic of inference applies to both.
Quantitative research features:
Uses numbers and statistical methods.
Relies on numerical measurements of phenomena.
Abstracts from specific instances to seek general descriptions or test causal hypotheses.
Seeks measurements and analyses easily replicable by others.
Qualitative research features:
By definition, does not rely on numerical measurements.
Often focuses on a small number of cases with intensive data collection (e.g., depth interviews, historical materials).
Aims for a rounded or comprehensive account of a particular event or unit.
Generates large amounts of information from a few cases; may be linked to case studies or area studies.
History and other disciplines can be analytical and systematic; the authors argue for breaking down false dichotomies between methods and recognizing a shared logic of inference.
Key aim: social science should be empirical and systematic, with inferences grounded in public, explicit procedures, not private insights.
The Science in Social Science
Four features characterize scientific research (as defined by the authors):
1) The goal is inference: to make descriptive or explanatory inferences from empirical information; descriptive detail is necessary but not sufficient.
2) The procedures are public: explicit, codified methods that can be judged, taught, replicated, and tested by others.
3) The conclusions are uncertain: inference is inherently imperfect; uncertainty must be estimated and reported.
4) The content is the method: the rules of inference are central, not the particular subject matter; science is ultimately about method and procedures.These four features imply that social science is a public, cumulative enterprise, where errors are pointed out and progress comes from transparent methods and shared standards.
The authors emphasize that not all questions of philosophical nature are empirical, but where empirical work aims to learn about the real world, the rules of scientific inference apply.
Complexity is acknowledged: social phenomena are often complex, but complexity should not derail the use of scientific methods. The goal is to generalize where possible while recognizing the role of unique, context-specific factors.
Examples cited to illustrate these ideas:
Martin (1992) uses ninety-nine cases for quantitative analysis and then six detailed case studies to clarify causal inferences.
Putnam et al. (1993) used a large-scale survey and interviews to gain deep, contextual knowledge about regional politics.
Dinosaur extinction and the Alvarez meteorite hypothesis illustrate how a unique event can still be studied scientifically via observable implications and testable consequences.
The Schumpeter-Einstein quote on certainty highlights the tension between certainty and empirical truth.
Major Components of Research Design
Social science research is best thought of as a dynamic, creative process that operates within a stable structure of rules of inference.
Researchers often begin with a design, collect data, and then revise their questions or theory; the process is iterative and non-linear.
Four analytical components are identified and commonly discussed in sections 1.2.1–1.2.4:
The research question
The theory
The data
The use of the data
In practice, qualitative researchers may begin with data (data-first), while quantitative work may begin with a theory or a question; nevertheless, each component should be considered in designing an empirical inquiry.
The authors present an idealized framework assuming unlimited time and resources, acknowledging that real-world research requires compromises.
1.2.1 Improving Research Questions
Questions originate from a mix of creative intuition and engagement with the literature; there are no hard-and-fast rules for choosing topics.
Karl Popper-like view cited: discovery contains an irrational or creative element; rules for choosing topics are informal.
Two criteria for a good research question:
1) It should be important in the real world (consequential for politics, society, or economics; affects many people; helps understand harmful or beneficial events).
2) It should make a specific contribution to an identifiable scholarly literature (advance verified scientific explanations or enable understanding where knowledge is sparse).How to contribute to the literature can take various forms, including:
1) Testing an important hypothesis lacking systematic study.
2) Re-examining an accepted hypothesis to confirm or refute it.
3) Resolving or contributing to a literature controversy.
4) Illuminating or evaluating unquestioned assumptions.
5) Highlighting an overlooked topic and conducting a systematic study.
6) Demonstrating applicability of theories/evidence from one literature to another.A tension exists between real-world relevance and long-term scientific value; researchers should aim to satisfy both criteria, but practical considerations may prioritize one over the other in the short run.
The authors favor a strategy of dialog with the literature: assess what is already known and refine questions to maximize explanatory potential.
The discussion emphasizes that research topics should be adaptable and that a project should be capable of yielding valid descriptive or causal inferences.
If a proposed topic cannot be shaped into a research project capable of valid inference, it should be revised or abandoned.
1.2.2 Improving Theory
A social science theory is a reasoned, precise speculation about the answer to a research question, including why the proposed answer is correct, and it implies multiple testable hypotheses.
Theories should be consistent with prior evidence; a theory that ignores existing evidence is unacceptable (Lieberson 1992; Woods and Walton 1982).
Four guidelines for improving theory (and a broader discussion of falsifiability):
1) Choose theories that could be wrong; include clear criteria for what evidence would prove the theory wrong.
2) Ensure falsifiability by generating as many observable implications as possible, enabling multiple tests across diverse data sets.
3) Be as concrete as possible in theory statements; explicit, testable predictions facilitate falsification.
4) Use parsimony judiciously; simple theories may have higher prior probability, but parsimony should not be imposed as a universal rule in the social sciences.The authors caution against ad hoc adjustments that artificially salvage a theory after data collection; any modifications should expand the theory’s applicability rather than restrict it to fit observed cases.
Two practical rules after data collection:
1) If a prediction depends on several variables, consider relaxing one condition to broaden the theory’s applicability for falsification.
2) Avoid restricting the theory to fit the observed data without seeking new data to test the revised version. If a theory fails for a country (or context), do not merely narrow its scope to save it.When data are unavailable or insufficient, it may be appropriate to propose future research designs or data collection efforts to decide whether the theory is correct; such speculative work should be clearly labeled as uncertain.
Pilot projects can be valuable to test ideas and refine data collection strategies before large-scale studies.
1.2.3 Improving Data Quality
Data quality is enhanced by systematically recording the data-generation process so that analyses can be evaluated for potential biases.
Key guideline: document the method by which data were generated (sample selection, questions asked, case selection rules, etc.).
The authors emphasize sharing data and methods to allow replication and evaluation; data sharing enhances scholarly credit and progress.
To improve inference, researchers should collect data on as many observable implications of the theory as possible, across diverse contexts and units; this increases leverage by testing the theory in multiple settings.
Strategies for expanding observable implications include disaggregating data to smaller units, examining related dependent variables, and designing alternative data collection contexts (e.g., lab experiments, cross-domain comparisons).
An example discussed: rational deterrence theory could be tested via multiple observable implications, not just direct threats; cross-domain analogies (industrial organization, game theory) can provide additional observable implications.
The discussion emphasizes that increasing data broadly typically strengthens inference, but data collection should be purposeful and theory-driven, not aimless.
1.2.4 Improving the Use of Existing Data
When data are flawed or limited, aim to generate unbiased inferences by considering averaging over many applications (unbiased across many datasets).
Important biases to watch: selection bias (choosing observations that support a theory) and omitted variable bias (excluding relevant controls).
Efficiency: use all available data and all relevant information within the data to improve inferences; disaggregate when possible (e.g., analyze smaller geographic units rather than only national aggregates).
Replicability: strive to make data, methods, and reasoning transparent so others can replicate or evaluate the study; qualitative replication can involve sharing field notes or interview protocols, while quantitative replication focuses on re-running analyses with the same data.
The authors discuss historical examples of replication (e.g., Middletown, Indiana) as a model for qualitative replication, though not always as extensive as the original study.
Even when replication is not feasible, providing sufficient detail about procedures enables evaluation and understanding of the study’s conclusions.
Themes of This Volume (1.3)
1.3.1 Using Observable Implications to Connect Theory and Data
Every theory should have observable implications—what we should be able to observe if the theory is correct.
Observable implications guide data collection and help distinguish relevant facts from irrelevant ones.
Theory and empirical work must be tightly connected; empirical investigations should be guided by theory, and theory should be tested by data.
Conclusions should be based on a strong link between theory and data, with observable implications being the vehicle for this link.
1.3.2 Maximizing Leverage
Leverage is the degree to which a theory explains many observable effects with few causal variables (or explains many effects with a few variables).
High leverage occurs when a simple, powerful explanation accounts for a large set of observations; low leverage occurs when many variables are needed to explain small effects.
Ways to increase leverage:
1) Improve the theory to yield more observable implications.
2) Improve the data to observe more implications.
3) Improve the use of data to extract more implications from existing data.Distinction from parsimony: maximizing leverage is a pragmatic objective tied to explanatory power and testability, not a general insistence on the world being simple.
Researchers should list all plausible observable implications across various data sets and contexts, including cross-level and cross-time observations, to test the theory more robustly.
Cautions against cross-level ecological inferences if the goal is to study individuals; however, if the theory operates at multiple levels, cross-level data can still be informative.
1.3.3 Reporting Uncertainty
All knowledge/inference is uncertain; both qualitative and quantitative research are subject to measurement error and probabilistic effects.
Qualitative researchers may face different kinds of measurement challenges than survey researchers; both require explicit reporting of uncertainty.
Good social scientists report uncertainty estimates for their inferences rather than presenting sweeping, certain conclusions.
Neustadt and May’s practical suggestion for policymakers: consider how much money you would wager on a conclusion and what odds you would assign; used to contextualize uncertainty in policy decisions.
1.3.4 Thinking like a Social Scientist: Skepticism and Rival Hypotheses
A cautious approach to causal claims: always consider alternative explanations, data accuracy, and potential confounding variables.
An illustrative example: Japan’s lower red meat consumption and heart attack rates; alternative explanations include data quality, dietary differences, genetics, lifestyle factors; consider potential reverse causality.
Causal inference is a process of continual refinement, testing, and considering rival hypotheses; strong inferences emerge gradually through successive approximations.
The authors emphasize skepticism and the iterative nature of causal inference as core to thinking like a social scientist.
Notable methodological positions and concepts (embedded throughout)
The four characteristics of science (public procedures, uncertain conclusions, content as method, inference goal) underpin the book’s argument for a unified approach to qualitative and quantitative research.
The book argues against an exclusive devotion to either quantitative or qualitative methods and instead promotes a design-based approach where reasoning, data, and theory are integrated.
Observable implications, leverage, and uncertainty are central themes for evaluating and improving research designs.
Examples and references throughout illustrate how these principles play out in real research (e.g., Martin 1992; Putnam et al. 1993; Alvarez et al. on dinosaur extinction; Gould 1989a).
The text acknowledges practical constraints (data limitations, time, resources) and encourages pilot studies, iterative design, and transparent reporting to maximize scientific contribution.
cases in Martin (1992), Italian regional councillors in Putnam et al. (1993), and figures in subsequent samples, and community leaders surveyed in 1983, among others.
Examples also include quantitative mentions like major wars in four hundred years, or the use of case-study replication over decades (e.g., Middletown studies). These numerical references serve to ground the discussion of methodology in concrete research practices.
End of Notes
Aims and Objectives
The primary aim of "Designing Social Inquiry" by King, Keohane, and Verba is to produce valid descriptive and causal inferences about social and political life through proper research design in the social sciences.
It emphasizes practical design choices rather than theoretical philosophy or specific data techniques (e.g., surveys, fieldwork, statistics).
A key objective is to connect quantitative and qualitative research traditions by applying a unified logic of inference to both, thereby bridging methodological divides.
The book also aims to ensure social science is empirical and systematic, with inferences publicly grounded in explicit procedures rather than private insights.
Main Argument
The central claim is that good research, whether quantitative or qualitative, derives from a single, underlying logic of inference. Differences between these two styles are considered mainly stylistic and methodological, not fundamental to the scientific process.
The unified logic of inference applies universally, arguing against exclusive devotion to either quantitative or qualitative methods in favor of a design-based approach that integrates reasoning, data, and theory.
Conclusions
Scientific research in social science is characterized by four features: the goal is inference, procedures are public, conclusions are uncertain, and the content is the method.
All knowledge and inference are inherently uncertain; both qualitative and quantitative research are subject to measurement error and probabilistic effects. Good social science requires reporting these uncertainty estimates.
Causal inference is an iterative process involving continual refinement, testing, and the consideration of rival hypotheses, with strong inferences emerging gradually.
Observable implications, leverage, and uncertainty are central themes for improving research designs, connecting theory and data effectively.
A skeptical approach and iterative testing of rival hypotheses are core to thinking like a social scientist.
Relevance
The principles apply broadly across political science and other disciplines such as sociology, anthropology, history, economics, and psychology.
It is also relevant to non-disciplinary areas, including legal evidence, education research, and clinical reasoning, wherever empirical work seeks to learn about the real world.
The book asserts that even complex social phenomena should not deter the use of scientific methods; the goal is to generalize and recognize unique context-specific factors.
Criticisms
The provided notes do not contain any specific criticisms of "Designing Social Inquiry." The text focuses on outlining the book's arguments, methodologies, and guidelines, rather than presenting external critiques.