knowt logo

Class 9 Bias 2020 Sp1

BIAS IN RESEARCH

STEPS OF RESEARCH

  • Get an idea, select a topic or problem, review literature

  • State the research question, write hypotheses, define variables

  • Select study design, sample parameters, measurement method

  • Determine data storage, analysis, budget, gain IRB approval

  • Conduct pilot study, analyze results, make adjustments

RESEARCH BIAS

  • Bias occurs due to limitations in study design or conduct

  • Uncertainty in results due to potential biases

  • Difficulty in determining the degree of bias influence

THREE TIMES TO AVOID RESEARCH BIAS

  • Study planning, execution, and analysis stages

STUDY PLANNING

  • Experimenter bias, design bias, selection/sampling bias

  • Consideration of variables, operationalization, recruitment

STUDY EXECUTION

  • Procedural bias, measurement bias, errors in data collection

  • Importance of consistent procedures, training, and calibration

STUDY ANALYSIS

  • Methods/procedures, data storage, statistical tests

  • Reporting all data, avoiding attrition bias and reporting bias

P-HACKING WITH BRIAN WANSINK

  • P-Hacking for statistically significant results

  • Wansink's study on choice architecture for eating habits

THE CANADIAN JOURNAL OF NEGATIVE RESULTS

  • Lack of publication for studies showing no effect

  • Negative impact on scientific progress and resources

FOOD FOR THOUGHT

  • Brian Wansink's controversial food research

  • Downfall and questions raised about scientific integrity

Page 21: The Wansink Dossier

  • Brian Wansink: high-profile researcher at Cornell University

    • World-renowned eating behavior expert

    • Keynote speaker, author of bestselling books

  • Engages in questionable research practices

    • Admitted to these practices himself

    • Increasing number of papers retracted or corrected

  • Still a professor at Cornell University and director of the Food and Brand Lab

Page 22: Retractions of Brian Wansink's Studies

  • Brian Wansink had 15 studies retracted

  • Cautionary tale on bad incentives in science

Page 23: Reporting Bias in Science News Cycle

  • Media reports may misrepresent study results

  • Importance of evaluating evidence behind scientific claims

Page 24: Spotting Bad Science

  • Sensationalized headlines misrepresent research findings

  • Unrepresentative samples lead to biased conclusions

  • Conflicts of interest can influence research outcomes

  • Correlation does not imply causation

  • Unsupported conclusions and problems with sample size

  • Tools for assessing bias in scientific studies

Page 25: Bias Assessment Tools

  • Cochrane Collaboration for systematic reviews

  • Tools like Downs & Black Tool for assessing different types of studies

Page 26: Downs & Black Tool

  • Designed to assess all types of studies in about 20 minutes

  • Assesses several types of bias in studies

Page 27: GRADE Assessment Tool

  • System for developing and presenting evidence summaries

  • Domains include Risk of Bias

CP

Class 9 Bias 2020 Sp1

BIAS IN RESEARCH

STEPS OF RESEARCH

  • Get an idea, select a topic or problem, review literature

  • State the research question, write hypotheses, define variables

  • Select study design, sample parameters, measurement method

  • Determine data storage, analysis, budget, gain IRB approval

  • Conduct pilot study, analyze results, make adjustments

RESEARCH BIAS

  • Bias occurs due to limitations in study design or conduct

  • Uncertainty in results due to potential biases

  • Difficulty in determining the degree of bias influence

THREE TIMES TO AVOID RESEARCH BIAS

  • Study planning, execution, and analysis stages

STUDY PLANNING

  • Experimenter bias, design bias, selection/sampling bias

  • Consideration of variables, operationalization, recruitment

STUDY EXECUTION

  • Procedural bias, measurement bias, errors in data collection

  • Importance of consistent procedures, training, and calibration

STUDY ANALYSIS

  • Methods/procedures, data storage, statistical tests

  • Reporting all data, avoiding attrition bias and reporting bias

P-HACKING WITH BRIAN WANSINK

  • P-Hacking for statistically significant results

  • Wansink's study on choice architecture for eating habits

THE CANADIAN JOURNAL OF NEGATIVE RESULTS

  • Lack of publication for studies showing no effect

  • Negative impact on scientific progress and resources

FOOD FOR THOUGHT

  • Brian Wansink's controversial food research

  • Downfall and questions raised about scientific integrity

Page 21: The Wansink Dossier

  • Brian Wansink: high-profile researcher at Cornell University

    • World-renowned eating behavior expert

    • Keynote speaker, author of bestselling books

  • Engages in questionable research practices

    • Admitted to these practices himself

    • Increasing number of papers retracted or corrected

  • Still a professor at Cornell University and director of the Food and Brand Lab

Page 22: Retractions of Brian Wansink's Studies

  • Brian Wansink had 15 studies retracted

  • Cautionary tale on bad incentives in science

Page 23: Reporting Bias in Science News Cycle

  • Media reports may misrepresent study results

  • Importance of evaluating evidence behind scientific claims

Page 24: Spotting Bad Science

  • Sensationalized headlines misrepresent research findings

  • Unrepresentative samples lead to biased conclusions

  • Conflicts of interest can influence research outcomes

  • Correlation does not imply causation

  • Unsupported conclusions and problems with sample size

  • Tools for assessing bias in scientific studies

Page 25: Bias Assessment Tools

  • Cochrane Collaboration for systematic reviews

  • Tools like Downs & Black Tool for assessing different types of studies

Page 26: Downs & Black Tool

  • Designed to assess all types of studies in about 20 minutes

  • Assesses several types of bias in studies

Page 27: GRADE Assessment Tool

  • System for developing and presenting evidence summaries

  • Domains include Risk of Bias