HARking and P- Hacking

Introduction to Data Ethics

  • Key Themes: Discusses the ethics of data research with a focus on two unethical practices: HARKing and p-Hacking.
  • Purpose: To highlight how these practices can mislead scientific reporting and the implications for research integrity.

HARKing (Hypothesizing After the Results are Known)

  • Definition: HARKing is the practice of changing or developing hypotheses after the results have been obtained. It presents an illusion that the researcher predicted the finding originally.
    • Origin of the Term: Introduced in the Chapter 1 Data Ethics feature (Kerr, 1998).
  • Consequences of HARKing:
    • It blurs the lines between confirmatory research (testing specific hypotheses) and exploratory research (gathering data without specific expected outcomes).
    • Can lead to misleading conclusions where researchers may retroactively justify a hypothesis based on results, thus misrepresenting the original investigative intent.
  • Example:
    • The xkcd comic strip example titled “Significant” illustrates HARKing. In this comic, scientists start with the hypothesis that “jelly beans cause acne” but only find a significant result for green jelly beans. They then create a dramatic report focusing solely on this result.
    • Punchline of Comic: Highlights potential media misinterpretation with a fictitious newspaper headline “GREEN JELLY BEANS LINKED TO ACNE!”
    • Ethical Scientific Reporting: Ethical researchers should report original hypotheses and all related findings regardless of their statistical significance to maintain transparency.

Publication Bias in Science

  • Publication Likelihood: Research indicates that journals favor publishing studies that reject the null hypothesis over those that do not (Andrews & Kasy, 2017).
  • Hypothesis Confirmation:
    • Academic journals are inclined towards studies confirming preconceived hypotheses, increasing the pressure to engage in practices like HARKing.

p-Hacking

  • Definition: p-Hacking combines the symbol 'p' (referring to the alpha level, typically 0.05) with the term 'hacking,' indicating manipulative practices to achieve significant results.
  • Characteristics of p-Hacking:
    • Researchers may decide criteria for removing outliers or other data manipulations only after looking at initial analyses.
    • They might analyze data incrementally, stopping only when significant results are found instead of sticking to predetermined guidelines.
    • Collecting multiple outcome variables (e.g., not just acne but also other health indicators like indigestion) but only reporting those that are statistically significant.
    • Utilization of various predictor variables (e.g., different colors of jelly beans) while only reporting the predictors that yield significant findings.
  • Implications of p-Hacking:
    • Creates a false sense of significance in research findings and systematically undermines the reliability of reported results.
    • The need for preregistration of studies to counteract these unethical practices is emphasized.

Ethical Recommendations for Reporting Results

  • Clear distinction must be made between original hypotheses and others that may develop post-results.
  • Report all variables and conditions of the experiment transparently to avoid misleading interpretations of surprising findings.
  • Recommendations supported by literature on research practices (Hollenbeck & Wright, 2017; Simmons et al., 2011).

Conclusion

  • Both HARKing and p-Hacking illustrate critical challenges in maintaining ethical standards in scientific research.

  • There is a growing acknowledgment within the scientific community for the need for transparency and accountability in research practices to restore public trust in scientific findings.

  • Interactive Component: Engage in the practice of p-Hacking through a simulation study available on FiveThirtyEight for an experiential understanding of the topic.