Scientific Method in Psychology: Overview

• The scientific method is broadly similar across sciences (biology, chemistry, material sciences) and also applies to psychology and social sciences.
• Goal: maximize objectivity and accuracy, recognizing that researchers are human and not perfectly objective.
• Core process ( cyclical and iterative ):
  • Form a research question of interest.
  • Develop predictions or hypotheses about how the world works.
  • Ground these in prior research and the current state of the field.
  • Test hypotheses by collecting data via rigorous methods.
  • Analyze the data to draw conclusions about how the world works.
  • Communicate findings so others can use, critique, or replicate them.
• Before conducting research, ethical and respectful study of people is required (human subjects research).
• Foundational ethics frameworks and governance:
  • Belmont Report (developed in the 1970s) guides ethical research with human participants.
  • Federal governance comes from bodies like Health and Human Services (HHS).
  • Institutional Review Board (IRB): reviews all research at a given institution to ensure ethical conduct and protection of human rights.
  • Guidelines apply to both federally funded and non-federally funded research at universities and research institutes.
• Core ethical pillars (brief): informed consent, confidentiality, and debriefing.
  • Informed consent: participants must understand what they are getting into; information presented at an appropriate developmental level; participants can withdraw at any time without penalty.
  • Confidentiality: data are kept private; responses may be anonymized or identified only by codes; data breach protections.
  • Special concerns for vulnerable populations (infants, children, prisoners, pregnant people and unborn fetuses).
  • Debriefing: after participation, researchers disclose the study’s purpose and methods; particularly important when deception is used to prevent bias (e.g., social desirability bias).
• Deception and social desirability bias:
  • Deception may be used to reduce bias, but must be followed by thorough debriefing.
  • Social desirability bias: participants may report what they think sounds better rather than what they actually think/feel.
• Ethical considerations in lifespan developmental research:
  • Children require additional protections; legal guardians provide consent for minors; assent (child agreement) is sought when possible.
  • Distress-inducing procedures are carefully considered and justified as part of everyday experiences (e.g., attachment studies).
  • Pregnant people and unborn fetuses require added protections to safeguard health and well-being.
• Historical context: ethical guidelines exist because past research failures were harmful; current guidelines aim to prevent such abuses.
• Reconnecting to the scientific method in practice:
  • Start with a broad topic of interest; identify gaps in knowledge; ensure feasibility and ethics; replicate where possible to strengthen evidence.
  • Understand different study designs and how they impact validity.

Key Concepts in Validity and Study Design

• Validity: the extent to which a study or measure reflects what it intends to measure.
  • Ecological validity: the accuracy with which findings reflect real-life processes and contexts.
  • Internal validity: the likelihood that observed effects reflect a true causal relationship rather than confounds.
  • Trade-off: some designs improve ecological validity but reduce internal validity, and vice versa. Different questions may require different balances.
• The claim that there is only one design that establishes causality: experimental designs.
• Descriptive research: aims to describe what is happening (often debated as a separate category from correlational and experimental).
• Correlational designs:
  • Study two factors as they occur naturally, without manipulation.
  • High ecological validity, but low internal validity (correlation does not imply causation).
  • Possible third variables or bidirectional influences.
• Experimental designs:
  • Manipulate an independent variable (IV) and measure a dependent variable (DV).
  • Conducted under controlled conditions (often in a lab).
  • Higher internal validity, enabling causal inferences; ecological validity can be lower due to artificial settings.
  • Variables: IV (manipulated) and DV (measured).
  • Control of extraneous variables to isolate the effect of the IV.
• Random assignment:
  • Participants are randomly assigned to two or more groups to balance individual differences across groups.
  • Example: IV = eating before an exam (eat vs. do not eat); DV = exam performance.
  • Random assignment helps ensure that confounding variables are evenly distributed across groups.
• Field experiments vs laboratory experiments:
  • Field experiments: conducted in natural settings; greater ecological validity but less experimental control.
  • Lab experiments: conducted in controlled environments; higher internal validity but potentially lower ecological validity.
• Interventions and longitudinal designs:
  • Intervention programs can be tested in experiments or quasi-experimental designs.
  • Longitudinal studies track the same participants over time to observe change.
• Quasi-experiments and natural experiments:
  • Quasi-experiments: lack random assignment; groups naturally differ (e.g., different cultures or policy changes).
  • Natural experiments: events outside the researcher’s control are used to study effects (e.g., policy changes, COVID-19 as a natural catalyst).
• Mixed-methods and meta-analysis:
  • Mixed designs combine elements from different designs to suit the question.
  • Meta-analysis pools data from multiple studies to estimate overall effects and improve generalizability.
• Cross-cultural and diverse sampling:
  • Aim to move beyond WEIRD samples to improve representativeness.
  • WEIRD acronym (extremely common in psychology) stands for White, Western, Educated, Industrialized, Rich, Democratic; researchers are increasingly cautious about overgeneralizing from WEIRD samples.
  • Strategies to diversify samples include oversampling underrepresented groups, collaborating across institutions, and using national datasets for representative samples.
• Cross-cultural research cautions:
  • Avoid lumping diverse groups into broad labels (e.g., “Asians,” “Latinos”) as if they are homogeneous.
  • Historically, white European American samples often served as a default comparison group, which is now discouraged.

Developmental Designs: How to Study Change Over Time

• Cross-sectional design:
  • Recruit participants of different ages at the same time.
  • Pros: quick, relatively inexpensive; provides a snapshot across ages.
  • Cons: cohort effects—age groups may differ due to historical/cultural experiences rather than developmental processes.
  • Example discussed: COVID-19 pandemic affecting children at different ages could confound age-related development with pandemic experiences.
• Longitudinal design:
  • Follow the same group of individuals over time.
  • Pros: controls for cohort effects; reveals developmental trajectories.
  • Cons: time-consuming, expensive; cross-generational generalizability limited (e.g., findings may not generalize to future cohorts); attrition can be an issue.
• Cross-sequential design:
  • Combines cross-sectional and longitudinal approaches.
  • Recruit cohorts at different ages and follow them for a shorter period.
  • Pros: balances time and cohort concerns.
  • Cons: more complex logistics and analysis.
• Microgenetic design:
  • Intensive, moment-to-moment study of change over a short period when a developmental change is occurring.
  • Focuses on mechanisms and processes driving change (e.g., puberty, rapid skill acquisition).
  • Typically produces a large amount of data over a brief window.

Sampling and Representativeness in Developmental Research

• Participant selection and representativeness:
  • Researchers often focus on a narrower age range or specific cultural groups relevant to the question.
  • Samples should be representative of the population of interest; no single study can include every individual.
  • Within-group diversity matters (e.g., even within Chinese Americans there are varied experiences).
• Historical biases in samples:
  • Early psychology studies often used male college students and white, middle-to-upper-class participants; now recognized as problematic and insufficient for generalization.
• WEIRD populations and beyond:
  • WEIRD: White, Western, Educated, Industrialized, Rich, Democratic; criticized for not representing global diversity.
  • Researchers respond by oversampling underrepresented groups, collaborating across universities, using large national datasets (e.g., NICHD data), and conducting cross-cultural research.
• Cross-cultural research practices:
  • Use culturally appropriate measurement and language; avoid assuming identical constructs map across cultures.

Data Collection Methods and Measurement in Developmental Research

• Surveys and questionnaires:
  • Structured measures with fixed items; can be paper, online, or self-report.
  • Risks: social desirability bias; responses may be influenced by wording and context.
• Structured vs unstructured interviews:
  • Structured: fixed questions; easier to compare across participants.
  • Unstructured: flexible, clinical approach; tailored to participant; useful for capturing individual differences, especially with children.
  • Downsides: harder to compare; potential for confirmation bias or leading questions.
• Observational methods:
  • Naturalistic observations: in participants’ natural environments (e.g., video-recorded family dinners at home); participants know they are observed, with data kept confidential.
  • Structured observations: conducted in labs or controlled environments; scenarios are set up and video-recorded for later coding.
  • Behavior coding: trained researchers code specific behaviors from video or live observation.
• Attachment research example (structured observation):
  • Mary Ainsworth’s Strange Situation protocol used to assess primary caregiver attachment.
  • Typical procedure: mother-child play; mother leaves the room; a “stranger” enters; mother returns; child’s reactions coded (e.g., crying, clinging).
• Physiological and biological measures:
  • EEGs (electroencephalography) with head-mounted electrodes;
  • MRI (brain imaging);
  • Saliva or blood samples; hormone levels.
  • These measures can provide objective data about physiological processes related to development.
• Practical and ethical considerations for measures:
  • Measures must be valid (measure what they are intended to measure).
  • Some measures (especially surveys) can be influenced by wording and administration; validity concerns arise if items don’t reflect the construct well.
  • Reliability: consistency of measurements over time (test-retest) and across observers (inter-rater reliability).
• Reliability specifics:
  • Test-retest reliability: stability of a measure across time when the underlying trait is stable.
  • Inter-rater reliability: agreement among different coders or observers when coding behavior.
• Data privacy and ethics in measurement:
  • Ensure confidentiality and control access to sensitive data; deidentify data with codes instead of names.
  • Be mindful of potential harms from data disclosure (e.g., sensitive personal or legal information).

From Data to Analysis: Reading and Reporting Results

• After data collection, the next step is data analysis.
• The instructor briefly notes that statistical analysis is a large topic and not covered in depth in this session.
• In practice, analysis involves choosing appropriate statistical tests based on design (correlational, experimental, longitudinal, etc.), checking assumptions, and interpreting results in light of validity and reliability concerns.

Key Takeaways and Practical Implications

• Always ground questions in existing literature and identify knowledge gaps that are feasible and ethically permissible to study.
• Consider both internal and ecological validity when designing a study; sometimes you trade one for the other depending on the research question.
• Understand the strengths and limitations of different designs: correlation for naturalistic validity, experiments for causal inference, and longitudinal approaches for development over time.
• Always plan sampling carefully to ensure representativeness and consider WEIRD biases; diversify samples when possible.
• Use multiple methods of data collection to triangulate findings (surveys, interviews, observations, physiological measures) where appropriate.
• Ethics are not optional: obtain informed consent/assent, protect confidentiality, minimize risk, and provide debriefing; special protections exist for vulnerable populations.
• When reporting results, link back to feasibility, ethical considerations, and the broader implications for practice, policy, and further research.

Equations, Symbols, and Notable Formulas (LaTeX)

• Correlation vs causation concept:
  r
  eq ext{causation}
• Independent variable (IV) and dependent variable (DV) relationship (conceptual):
  ext{DV} = f( ext{IV}, ext{controls})
• Random assignment concept (probabilistic grouping):
  P( ext{Group } g) = rac{1}{k}, ext{ for } g = 1,
  ightarrow k
• Validity definitions (conceptual forms):
  ext{Ecological Validity} = ext{real-world applicability of findings}
  ext{Internal Validity} = ext{confidence that observed effects are due to IV}
• Cross-sectional vs longitudinal notation (conceptual):
  ext{Cross-sectional}: ext{ multiple ages at one time}
  ext{Longitudinal}: ext{ same individuals over time}
• Microgenetic design (concept):
  ext{Microgenetic design}
  ightarrow ext{intense data during rapid development}
• WEIRD acronym (conceptual representation):
  ext{WEIRD} = igl{ ext{White}, ext{Western}, ext{Educated}, ext{Industrialized}, ext{Rich}, ext{Democratic} igr
  floor