Notes: Research Methods (Empirical Methods)

The Scientific Method

  • The scientific method in social psychology involves: observe data, develop theories or explanations, generate testable hypotheses, design a study, collect and analyze data, interpret results, refine or replicate theory, and publicize data.

  • A testable prediction is often implied by a theory; hypotheses are testable predictions derived from theories.

  • Process sequence (as presented in the slides):

    • Observe “interesting” data

    • Develop theories or explanations

    • Develop hypotheses

    • Test the Hypothesis: design a study, collect and analyze data

    • Interpret the results

    • Refine or replicate theory and publicize data

  • Social psychology is framed as scientific because it systematically tests theories about human behavior using empirical data.

Research Designs

  • Three broad designs:

    • Observational (descriptive): observing and describing behavior

    • Correlational (predictive): studying relations between variables

    • Experimental (causal): studying causal relations between variables

  • Operational definitions are crucial: specify procedures for manipulating or measuring a conceptual variable (e.g., aggression, helping behavior).

  • Variables:

    • Independent (predictor) variable (IV): what the researcher expects to cause change; may be measured (correlational) or manipulated (experimental)

    • Dependent (outcome) variable (DV): what is measured; expected to change in response to the IV

  • Summary of method focus and questions:

    • Observational: Description — what is the nature of the phenomenon?

    • Correlational: Prediction — from knowing X, can we predict Y?

    • Experimental: Causality — is variable X a cause of Y?

Operational Definitions and Variables

  • Operational definitions specify concrete procedures for manipulating or measuring a conceptual variable (e.g., aggression, helping behavior).

  • Importance: ensures clarity, reliability, and replicability across researchers and settings.

OBSERVATIONAL RESEARCH (DESCRIPTIVE)

  • Define: observe and describe behavior occurring naturally, in natural settings.

  • Examples:

    • Festinger & the cult

    • Observing kids at school

    • Public bathroom hand-washing studies

  • Methods of describing behavior:

    • Ethnography: description from an “insider’s point of view”

    • Archival analysis: researcher examines accumulated documents

  • Observational research process (typical steps):

    • Research Question

    • Behaviors concretely defined before observation

    • Observer systematically records behaviors

    • Accuracy of observer assessed via interjudge reliability

  • Archival analysis example topics:

    • PornHub traffic during a nuclear scare (archival data visualization)

    • Scraping social media data; sentiment analysis (millions of Twitter messages) to assess mood variation

  • Archival examples in slides:

    • “Pulse of the Nation: U.S. Mood Throughout the Day inferred from Twitter”

  • Strengths and limitations:

    • Strengths: study behavior in natural context; results easy to disseminate

    • Limitations: some behaviors are difficult to observe; participant reactivity; researcher subjectivity; uncontrolled settings; limited in predictive/explanatory power

CORRELATIONAL RESEARCH

  • Purpose: identify relationships between naturally occurring variables; not manipulated by the researcher.

  • Common example: surveys measuring relationships between variables (e.g., smoking and lung cancer; self-esteem and relationship quality; pornography consumption and sexism).

  • Strengths: can include many participants; can collect data on many variables.

  • Limitations: susceptibility to question effects; samples may not be representative; responses may be biased (social desirability).

  • Random sampling concept (in surveys): ensures the sample represents the population; example discussed: 1936 presidential election and the Literary Digest failure due to sampling bias (telephone/car ownership bias).

  • Random Sampling (explicit): critical for representativeness; helps generalize findings.

  • Correlation coefficient: r

    • A statistical technique that assesses how well you can predict one variable from another

    • The sign indicates the direction of the relationship:

    • Positive: variables move in the same direction (as X increases, Y increases)

    • Negative: variables move in opposite directions (as X increases, Y decreases)

    • Range: r ext{ ranges from } -1.00 ext{ to } +1.00

    • Examples: positive correlation, negative correlation shown conceptually

  • Statistical significance in correlational findings:

    • Significance threshold commonly set at p < 0.05 his indicates a less-than-5% chance that the observed effect is due to random variation in the sample

    • Relationship between effect size and p-value: a correlation of |r| = 0.50 is generally stronger (larger effect) than |r| = 0.10, even if both are statistically significant

    • Example notes: a correlation of r = 0.95 with p = 0.01 is extremely strong; r = 0.45, p = 0.03 indicates a moderate relationship with low probability of being due to chance

  • Perils of inferring causation from correlation:

    • Correlation does not equal causation

    • Possible third variables (confounds) that influence both X and Y

    • Directionality problem: unclear which variable influences the other

    • Examples in slides: birth control type and STI risk; substance use and abortion rates; pornography use and sexism (as correlational associations rather than proven causal links)

  • Strengths and limitations recap:

    • Strengths: reflects real-world factors that cannot be ethically manipulated; can handle large datasets; broad variable coverage

    • Limitations: cannot prove causation; susceptible to confounds; questions of measurement validity rely on survey quality

RANDOM SAMPLING (SURVEY RESEARCH)

  • Emphasis on representativeness of the population sample

  • Classic historical example: 1936 Literary Digest poll predicting a landslide defeat for FDR due to sampling bias (only those with telephones or cars were polled, skewing political leanings)

  • Practical takeaway: representative sampling is critical for generalizing survey results

EXPERIMENTAL RESEARCH

  • Purpose: determine cause-and-effect relationships

  • Core components:

    • Random assignment: equal chance for participants to be in any experimental group or control group; helps minimize confounds

    • Manipulation of one or more Independent Variables (IVs)

    • Measurement of Dependent Variables (DVs)

    • Control over the experimental environment

  • How experiments establish causality:

    • IV is manipulated, DV is measured

    • Extraneous variables controlled (often through random assignment)

  • Key terms:

    • Independent Variable (IV): manipulated by the researcher; levels (two minimum)

    • Dependent Variable (DV): measured by the researcher; hypothesized to change due to the IV

  • Experimental design types:

    • Between-subjects design: different participants in each group/treatment; groups compared to assess the effect of the IV

    • Within-subjects design: same participants experience multiple groups/treatments

  • Example illustrations:

    • Aggression in 4-year-olds after viewing 30-minute videos of Barney vs. Teenage Mutant Ninja Turtles; IV, DV defined

    • Adolescent condom use after peer-based, counselor-based, or no-information intervention; IV, DV defined

  • Random assignment rationale:

    • Prevents pre-existing variables (e.g., intelligence, hunger) from systematically biasing group differences

  • Experimental designs and outcomes:

    • Multiple groups differing on DV to assess whether the IV caused differences

    • Statistical significance concept applies to determine whether group differences are unlikely due to chance

  • Statistical significance recap:

    • The probability level for an effect being due to chance is less than 0.05 for an effect to be considered statistically significant

  • Strengths and limitations:

    • Strengths: high control; ability to infer causality

    • Limitations: some variables cannot be manipulated; ethical constraints; artificiality may reduce realism

  • Internal validity vs external validity:

    • Internal validity: confidence that observed effects are caused by the IV (control for confounds)

    • External validity: generalizability to other people, settings, IVs, and DVs; randomized samples, replication, and field experiments contribute to external validity

  • The basic dilemma:

    • Tradeoff between internal and external validity; the best practice is not to try to maximize both in a single experiment; instead, balance across multiple studies and designs to build a robust evidence base

EVALUATING EXPERIMENTAL RESEARCH: VALIDITY

  • Internal validity: extent to which you can draw causal conclusions

  • External validity: extent to which findings generalize to other people, settings, times, and conditions

  • Strategies to enhance validity:

    • Use random assignment to control for confounds

    • Conduct replication studies

    • Include field experiments to improve ecological validity

    • Consider representative samples when possible

ETHICS IN PSYCHOLOGICAL RESEARCH (DISCUSSION)

  • Two goals often in conflict:

    • Create experiments that resemble real-world settings while maintaining rigorous control

    • Avoid causing participants undue stress or harm

  • Who decides? Responsible researchers and Institutional Review Boards (IRBs) evaluate costs and benefits

  • Respecting people: core ethical principles

    • Respect for persons

    • Beneficence

    • Justice

    • Truthful reporting and accurate credit/sharing of data

  • Key ethical guidelines (APA code of ethics):

    • Informed consent

    • Freedom from coercion

    • Protection from harm

    • Risk-benefit analysis

    • Deception and debriefing guidelines

    • Confidentiality

  • Informed consent and deception:

    • Informed consent: participants agree to participate with full knowledge of study nature; sometimes not fully feasible in some social experiments

    • Deception: allowed in some cases but must be justified; participants should be debriefed after the study to reveal true purpose and events

  • Institutional Review Board (IRB):

    • Ensures safety and dignity of research participants

    • Includes at least one scientist, one nonscientist, and one person unaffiliated with the institution

    • Reviews all proposals and approves studies before research is conducted

  • Notable ethical debates and cases:

    • Facebook emotion contagion study (Emotion contagion via News Feed) raised questions about informed consent and data use; concerns about privacy and consent under the Common Rule

    • Open discussion about the proper standards for consent, especially when data come from private platforms

  • Animal ethics in research:

    • All procedures involving animals must be supervised by trained researchers

    • Minimize discomfort and pain; justify use of animals when alternative methods are unavailable or when research value justifies it

    • Use anesthesia where appropriate

  • Ethics of publishing and peer review:

    • Peer review is typically a 3-person panel; iterative process with revision requests

    • Potential problems: slow process; possible failure to detect errors; susceptibility to spoofed or low-quality work (e.g., SCIgen nonsense papers; single-blind reviews over time)

    • Open science movements promote transparency, replication, and data sharing; open science badges signal rigor and transparency

  • Respecting truth in science:

    • Honest reporting of methods and results; credit and data sharing; ethical reporting of data

    • Notable misconduct cases (e.g., Stapel) highlighted the need for replication, verification, and whistleblowing; many papers were scrutinized or rejected after falsified data were revealed

  • Publishing and replication culture:

    • Reproducibility projects and multi-lab replication efforts (e.g., Many Labs, Center for Open Science) aim to test the robustness of high-profile studies

    • Open science movement advocates for preregistration, data sharing, and transparent reporting; adoption of badges for rigor and transparency

READING AN EMPIRICAL ARTICLE (HOW TO READ)

  • Don’t be intimidated by jargon or statistics; approach in manageable bites

  • Strategy for reading:

    • Abstract first to orient

    • Then the discussion (future directions) for context

    • Methods next to understand designs, analyses, measures, and procedures

    • Focus on flaws in methods; analyze analysis techniques and designs for replication

    • If planning your own project, study the methods section closely for operationalization and procedures

  • Practical tips for reading:

    • Start with the abstract and the first and last paragraphs of the introduction

    • Read the first and last paragraphs of the discussion to grasp big-picture contributions and future directions

    • Don’t get bogged down in statistics; derive the meaning from prose and figures

    • Imagine being a participant in the study to understand procedures and potential questions you would have

  • Time management:

    • Allocate time for a thorough read and margin notes; may require multiple passes

  • Purpose of reading:

    • For course work: focus on introduction and discussion for theory and implications; for a statistics class: focus on methods and results

  • Anatomy of an empirical article (core sections):

    • Abstract: brief summary including hypotheses, methods, results, implications

    • Introduction: background, leading to hypotheses, and theoretical framing

    • Methods: participants, measures, procedures, design; step-by-step for replication

    • Results: whether hypotheses were supported; interpretation of data; prose if tables/figures are intimidating

    • Discussion: implications, weaknesses, and future directions; how the study extends or challenges existing theory

  • Core idea: get the big picture first, then drill down into the specifics as needed

  • Reading and critical thinking checklist:

    • Identify the source of hypotheses and related literature

    • Assess whether methods adequately test the hypotheses

    • Evaluate the interpretation of results relative to the data

    • Consider limitations and alternative explanations

    • Reflect on how the study contributes to broader theory and real-world applications

  • Practical note on reading practice:

    • For future work, begin with the abstract, skim the intro/discussion for the main claims, then read methods for how those claims were tested, and finally examine results to see the data backing the conclusions

  • Final step: one last read-through after a break to solidify understanding and identify any new insights or questions

KEY CONCEPTS AND FORMULAE (RECAP WITH LATeX)

  • Correlation coefficient:

    • r = rac{ ext{Cov}(X,Y) }{ ext{SD}(X) imes ext{SD}(Y) } = rac{ rac{1}{n} ext{Cov}(X,Y) }{ }

    • In practice, use the standard Pearson correlation formula:
      r = rac{ ext{cov}(X,Y) }{ \sigmaX \sigmaY } = rac{ rac{1}{n} igg( extstyle\sum{i=1}^n (Xi - ar{X})(Yi - ar{Y})igg) }{ igg( rac{1}{n} extstyleigg( extstyle\sum{i=1}^n (Xi - ar{X})^2igg)^{1/2}igg) igg( rac{1}{n} extstyleigg( extstyle\sum{i=1}^n (Y_i - ar{Y})^2igg)^{1/2}igg) } }

  • Possible values and interpretations:

    • Positive correlation: r > 0

    • Negative correlation: r < 0

      Magnitude indicates strength: larger |r| means stronger linear relationship

  • Statistical significance threshold commonly used: p < 0.05

  • General note on effect size vs significance:

    • A correlation of |r| = 0.50 is a larger effect than |r| = 0.10, even if both are statistically significant at given sample sizes

  • Internal vs external validity (conceptual definitions):

    • Internal validity: confidence that the IV caused changes in the DV, free from confounds

    • External validity: generalizability of findings to other people, settings, times, and IVs/DVs

  • Definitions (brief recap):

    • Independent Variable (IV): manipulated by the researcher; levels indicate the presence/absence or dose of the manipulation

    • Dependent Variable (DV): measured by the researcher; expected to change due to the IV

  • Ethical and methodological highlights:

    • Informed consent, deception/debriefing, confidentiality

    • IRB oversight; open science practices; replication projects; data sharing

    • Publication integrity; prevalence of open science badges signifying rigor and transparency

  • Notable study references mentioned in the transcript (for context):

    • Donnerstein & Berkowitz (1981) – experimental design on pornography and aggression

    • Kramer, Guillory, Hancock (2014) – massive-scale emotional contagion via Facebook (PNAS) with subsequent editorial concerns and corrections

    • Stapel – data fabrication scandal highlighting replication and integrity concerns

    • Center for Open Science and Many Labs – open science replication initiatives

  • Practical implications and real-world relevance:

    • Understanding the difference between correlation and causation prevents misinterpretation of everyday findings

    • Ethical practices protect participants and maintain trust in science

    • Open science and replication strengthen the reliability of psychological findings

    • Reading empirical articles effectively enables building on existing knowledge for exams, research design, and critical thinking