The Persistent Sampling Bias in Developmental Psychology

Background & Core Definitions

• WEIRD concept
  • Acronym for Western, Educated, Industrialized, Rich, and Democratic societies.
  • Coined to capture the overrepresentation of such populations in psychological data sets.
• Culture (Kashima & Gelfand, 2012)
  • “A set of meanings or information that is non-genetically transmitted from one individual to another, shared within a population, enduring for generations.”
• Sampling diversity
  • Inclusion of participants from a broad range of sociocultural contexts in order to obtain generalizable findings.
• Othering
  • Treating WEIRD samples as the norm and non-WEIRD samples as deviations; rooted in feminist and critical race traditions (Ahmad, 1993; Fine, 1994).

Key Arguments & Thesis

• Developmental psychology is heavily skewed toward WEIRD samples, threatening the validity, reliability, and universality of its theories.
• Despite >$3000$ citations of Henrich et al. (2010) and years of discussion, empirical practices have barely changed.
• Ignoring cultural variation is “neglectful at best and bad science at worst.”
• Researchers must acknowledge potential cultural specificity of findings, or actively test for cross-cultural invariance.

Empirical Evidence of Sampling Bias

• Arnett (2008) baseline
  • $96\%$ of participants from Western industrialized countries; $68\%$ from the USA.
  • $99\%$ of first authors at Western institutions; $73\%$ USA-based.
• New audit (Nielsen et al., 2017)
  • Journals surveyed (2006-2010): Child Development, Developmental Psychology, Developmental Science.
  • Total articles: $1582$.
  • Geographic distribution of participants:
  • USA: $57.65\%$
  • Other English-speaking (UK, Canada, Australia, NZ): $17.95\%$
  • Non-English Europe: $14.92\%$
  • Israel/Middle East: $1.07\%$
  • Asia: $4.36\%$
  • Central & South America: $0.70\%$
  • Africa: $0.63\%$
  • Non-human primates: $1.45\%$ (exceeds African human data!)
  • Unspecified: $0.95\%$
  • Participants counted
  • USA: $633,775$ individuals (over twice the rest of world combined).
  • Non-WEIRD total (Central/South America, Africa, Asia, Middle East & Israel): < $3\%$ of all human participants while representing ≈ $85\%$ of global population.
  • Author affiliation (1691 articles including reviews)
  • USA: $61\%$
  • Other English-speaking: $20\%$
  • Non-English Europe: $15\%$
  • Asia + Israel: $4\%$
  • Central/South America: 2 articles
  • Africa & Middle East: 0 articles
• Temporal comparison
  • 2008 (361 articles) vs 2015 (383 articles)
  • WEIRD representation: $91.67\%$ → $92.37\%$ (virtually unchanged)
  • Non-WEIRD: $8.33\%$ → $7.63\%$

Illustrative Cross-Cultural Findings & Why They Matter

• Spatial cognition (Haun et al., 2006)
  • Hai//om children use geocentric strategies, while Western children prefer egocentric strategies.
  • If only Western data were collected, the field might declare egocentric search a “universal.”
• Infant–caregiver interaction
  • Variability in face-to-face play, object exploration, and maternal responses (Keller et al., 2004; Little et al., 2016).
• Folk-biology reasoning
  • Children’s biological categorization hinges on cultural experience with nature (Medin et al., 2010).
• Attention & Perception
  • Holistic vs analytic perception differs between East Asians and Westerners (Nisbett & Miyamoto, 2005).
• Language environment (“word gap”) interventions may not generalize beyond the U.S. dyadic interaction model (Hart & Risley, 1995; Rogoff et al., 1993).

Implications for Theory, Policy & Practice

• Theoretical distortion
  • Risk of labelling population-specific phenomena as species-wide universals.
• Policy misalignment
  • Educational or parenting programs grounded in WEIRD data may be ineffective or harmful elsewhere (Otto & Keller, 2014).
• Clinical & parental guidance
  • Advice on divorce outcomes, cognition under poverty, etc., varies cross-culturally (Fischer, 2007; Mani et al., 2013).
• Ethical responsibility
  • Scientists must avoid reinforcing hierarchies by presenting WEIRD norms as global standards.

Why the Bias Persists

• Practical constraints
  • Cost, time, and logistics of fieldwork in diverse locales.
  • Necessity to build long-term community trust before data collection.
• Institutional incentives
  • High-impact journals favor rigorous methodology; researchers in under-resourced contexts may lack access to training/equipment.
• Language & access barriers
  • Non-English scholars face hurdles in publishing and accessing pay-walled literature.
• Editorial & reviewer familiarity
  • Difficulty in judging studies that deviate from standard lab paradigms.

Recommended Actions & Path Forward

1. Explicit acknowledgment
   • When using homogeneous samples, authors should state potential limits to generalizability in their Discussion.
2. Encourage non-WEIRD studies
   • Journals and funders should prioritize research featuring under-represented populations.
3. Replication across cultures
   • Replicate classic findings in new cultural contexts to test universality.
4. Theory-driven comparisons
   • Frame cross-cultural work around clear hypotheses about cultural mechanisms.
5. Editorial diversification
   • Recruit editors/reviewers with field experience in diverse settings to fairly evaluate submissions.
6. Capacity building
   • Training workshops, open-access resources, and partnerships to empower scholars in low-resource regions.
7. Combat othering
   • Treat WEIRD samples as one cultural data point, not the baseline.

Philosophical & Ethical Considerations

• Science vs colonial mindset: Avoid perpetuating a view that WEIRD cognition = default human cognition.
• Reflexivity: Researchers must examine how their own cultural backgrounds shape assumptions, methodologies, and interpretations.
• Justice in knowledge production: Equitable representation of global populations is both a scientific necessity and moral imperative.

Connections to Broader Literature & Disciplines

• Comparative psychology & evolutionary biology (Herrmann et al., 2007; van Schaik & Burkart, 2011) stress the need for cross-species and cross-culture data.
• Anthropology and cultural psychology offer methods (e.g., ethnography, participant observation) for richer contextualization (Cole, 1996; Rogoff, 2003).
• Debates on external validity in experimental psychology (Anderson et al., 1999) intersect with calls for broader sampling.

Numerical Summary & Quick Reference

• Articles surveyed (2006-2010): $1582$
• Participant count (excluding Sweden registry):
  • USA $= 633\,775$
  • Rest of world $= 286\,321$
  • Unspecified $= 1\,959$
• Author affiliations (1691 articles): USA $61\%$, Other English $20\%$, Non-Eng. Europe $15\%$, Asia+Israel $4\%$.
• Change 2008 → 2015 in non-WEIRD coverage: $8.33\%$ → $7.63\%$ (no improvement).

Memorization Cues

• Think “633 thousand vs 286 thousand” → U.S. participants double the rest of world.
• “11 vs 10 vs 23” → Central/South America (11), Africa (10), Non-human primate studies (23).
• “3 & 85” rule → <$3\%$ of participants represent $85\%$ of global population.

Bottom line: Developmental psychology’s empirical core is built on a narrow slice of humanity. Addressing this sampling bias is essential to craft theories, policies, and practices that truly reflect the diversity of the human developmental experience.