Chapter 2 Notes — The Scientific Method in Sociology

Overview

• Sociologists use the scientific method to answer questions about how media, social networks, and other factors shape political beliefs and ideologies.
• The method is a systematic, organized series of steps that ensures objectivity and consistency in research, making findings replicable across researchers and contexts.
• Even if we never conduct a study, understanding the scientific method helps evaluate everyday claims that cite facts and data (e.g., crime is at an all-time high; one in every two marriages ends in divorce).
• Many common claims rely on statistics that can be misleading if not interpreted correctly. For example, comparing the number of marriages in a given year to the number of divorces in that same year does not measure the divorce rate for all marriages, since marriages and divorces are not the same set of marriages.
• Real-world divorce rates vary by age, income, education, and whether it’s a first or a subsequent marriage. The lifetime divorce rate for a first marriage is roughly $40\%$ to $45\%$ and has been declining.
• Public fear of crime can outpace actual data: violent crime rates have decreased since the 1990s even as fear remains high.
• Understanding the standards of scientific research helps evaluate assertions and resist misleading data or selective reporting.

The Role of Statistics and Data Quality

• Assertions like "crime is at an all-time high" or "one in two marriages ends in divorce" are common but may rely on selective reporting or misinterpretation of data.
• Proper data interpretation requires checking the source, understanding what is being measured, and whether the data reflect the population of interest.
• In sociology, researchers strive for objectivity, replicability, and clear measurement of concepts.

The Five Basic Steps of the Scientific Method (in Sociology)

• Step 1: Define the problem
  • State clearly what you hope to investigate and why it matters.
  • Example: Does going to college pay off in monetary terms?
• Step 2: Review the literature
  • Examine relevant scholarly studies to refine the problem, identify techniques for data collection, avoid duplication, and establish credibility.
  • Sources include peer-reviewed journal articles, books, and academic chapters; Google Scholar is a helpful tool.
  • Literature helps identify gaps and informs theory and methodology.
• Step 3: Formulate a hypothesis
  • A hypothesis is a speculative, testable statement about the relationship between two or more variables.
  • Variables are measurable traits or characteristics.
  • Types of variables:
  • Independent variable (IV): the cause or presumed influence (denoted as $X$).
  • Dependent variable (DV): the effect or outcome (denoted as $Y$).
  • Control variables (CV): factors kept constant to isolate the effect of the IV.
  • Example: Higher educational attainment (IV) is associated with higher income (DV).
  • In the example: $X = \text{education (years of schooling)}$, $Y = \text{income (past-year earnings)}$, and potential controls might include age, field of study, and geographic region.
• Step 4: Choose a research design and collect data
  • A sample is a subset of a larger population that is statistically representative.
  • Population vs. sample:
  • Population: the entire group of interest (e.g., all adults in the U.S.).
  • Sample: a smaller, manageable subset.
  • Sampling methods:
  • Random sample: every member of the population has an equal chance of selection.
  • Snowball or convenience sample: participants are recruited through referrals or accessible means.
  • Data sources example: The U.S. Census Bureau and the American Community Survey (ACS) provide large-scale data for analyses like the relationship between schooling and income.
• Step 5: Analyze data, draw conclusions, and report findings
  • Validity: the degree to which a measure accurately reflects the phenomenon being studied (accuracy).
  • Reliability: the degree to which a measure produces consistent results (consistency).
  • After analyzing, draw conclusions that either support or refute the hypothesis; discuss limitations and implications for future research.
  • The conclusion should also consider practical or theoretical implications (e.g., how findings relate to social inequality or policy).

Operational Definitions and the Importance of Measurement

• Abstract concepts in sociology (e.g., racism, power, deviance, happiness, poverty) are not directly observable.
• An operational definition provides a precise, testable way to measure a concept in a given study.
• Why this matters: without clear definitions, different researchers may measure the same concept in incompatible ways, leading to conflicting conclusions.
• Example from the ongoing study: define education as the number of years of schooling; define earnings as personal income received in the past year.
• The point: operational definitions tailor abstract concepts to measurable terms so they can be studied scientifically.

Hypotheses, Variables, and Causal Logic

• A hypothesis: a speculative statement about the relationship between two or more variables.
• Variables:
  • Independent Variable (IV): the presumed cause, denoted as $X$.
  • Dependent Variable (DV): the outcome, denoted as $Y$.
  • Control Variable (CV): factor held constant to test the IV's effect.
• Example: Hypothesis — higher education (IV) leads to higher income (DV).
  • IV: $X = \text{education (years of schooling)}$
  • DV: $Y = \text{income (past-year earnings)}$
  • CVs might include age, gender, locale, field of study, etc.
• Causal logic concepts:
  • Correlation: a change in one variable coincides with a change in another. Formally, a correlation exists when two variables move together, but correlation does not imply causation.
  • To establish causation, researchers consider:
  • Time order: the cause must precede the effect (does the IV occur before the DV?).
  • Non-spuriousness: there is no third variable that causes both the IV and DV.
• Example: TV watching and political knowledge
  • Could be a correlation: more TV watching co-occurs with lower political knowledge.
  • However, a third variable (e.g., overall cognitive ability) might influence both TV viewing and knowledge, obscuring a direct causal link.
• Demonstrative statements:
  • A correlation exists when a change in one variable coincides with a change in another.
  • A correlation does not guarantee causation; additional factors must be examined.

The Concept of a Sample and Representativeness

• A sample must be statistically representative of the population of interest.
• Why representative samples matter:
  • To ensure findings generalize to the larger population.
  • If the sample is biased, conclusions may be invalid for the population.
• Population and sample sizes are often constrained by practicality; studying the entire population is usually impossible.
• Common sampling types:
  • Random sample: gold standard for representativeness.
  • Snowball sample: recruits participants via referrals.
  • Convenience sample: recruits participants who are readily available.
• In the provided example, data from the Census Bureau, specifically the American Community Survey (ACS), would be used to construct the sample.

Validity, Reliability, and Data Quality

• Validity: the degree to which a measure reflects the actual concept being studied.
  • High validity means the data accurately represent the phenomenon.
• Reliability: the degree to which a measure yields consistent results across time and different conditions.
  • High reliability means repeated measurements give the same results.
• Data sources like the ACS are checked for accuracy and consistency to ensure both validity and reliability.
• The Census Bureau also assesses the consistency of responses (e.g., with growing online data collection) to maintain data quality.

Conclusion, Implications, and Authentic Inquiry

• The conclusion of a study summarizes findings and discusses whether the data support the original hypothesis.
• Even when data support the hypothesis, outliers and exceptions exist (e.g., some highly educated individuals earning less and some less-educated individuals earning substantial income).
• Findings can reveal social inequalities (e.g., unequal access to educational opportunities) and prompt further theoretical exploration (e.g., via conflict theory to examine disparities).
• Conclusions should inspire further research and potentially inform policy or broader theoretical perspectives.

Mini Field Experiment: Class Activity Prompt

• Instructions for a collaborative exercise:
  • Form groups of two to three students.
  • Identify a common social norm or behavior observed on campus or in public spaces.
  • Design a mini field experiment using the scientific method to explore that norm.
  • Write down the mini field experiment, including all group members' names, to receive participation credit.

Quick Reference: Key Terms and Concepts

• Scientific method: systematic, objective, and replicable process for investigating questions.
• Operational definition: precise, study-specific definition of a concept to enable measurement.
• Independent variable (IV): the presumed cause ($X$).
• Dependent variable (DV): the outcome ($Y$).
• Control variable (CV): a variable kept constant to isolate the effect of the IV.
• Correlation: a relationship where changes in one variable co-occur with changes in another, not implying causation by itself.
• Causation: one factor directly causing a change in another, demonstrated by time order and lack of confounding variables.
• Validity: accuracy of measurement for the concept.
• Reliability: consistency of measurement across trials.
• Population vs. sample: population is the whole group of interest; a sample is a manageable subset that should represent the population.
• Data sources: peer-reviewed sources, books, academic chapters; use tools like Google Scholar to locate credible literature.
• Example operational definitions (from the lecture):
  • $X = \text{education (years of schooling)}$
  • $Y = \text{income (earnings in the past year)}$
  • Potential controls: age, field of study, geographic region

Connecting to Real-World Relevance

• Understanding how statistics are gathered and interpreted helps evaluate media claims about crime, divorce, and other social phenomena.
• The scientific method provides a framework for assessing claims about political beliefs, media influence, and social inequality.
• Recognizing outliers and contextual factors is essential for a nuanced understanding of any empirical finding.