BIOL 4010U Lecture 5: Critically Evaluating Scientific Information

Vocabulary flashcards covering primary literature analysis, statistical significance vs effect size, publication bias, and science reporting issues from BIOL 4010U Lecture 5.

E.O. Wilson's Ideal Scientist

A concept stating that the ideal scientist "thinks like a poet, works like a clerk, and writes like a journalist."

BQMOC Analysis

A framework used to evaluate individual parts and figures of a scientific paper by analyzing Background, Question, Methods, Observations, and Conclusions.

Figure Facts

A data-centered template introduced by Round and Campbell (2012) to encourage undergraduates to systematically analyze and summarize figures when reading primary literature.

Gene V. Glass's View on Statistical Significance

The principle that statistical significance is the least interesting aspect of results, and that findings should be described in terms of measures of magnitude (how much a treatment affects subjects).

Pitfalls of Statistical Significance

Limitations of relying strictly on p-values, including an arbitrary cut-off of $95\%$ confidence, vulnerability to p-hacking, and restricted applicability to specific sample populations under specific conditions.

P-hacking

The misuse of data analysis techniques to manipulate or search through data until patterns emerge that can be reported as statistically significant.

Type I Error

A false positive error occurring when a researcher incorrectly rejects a true null hypothesis.

Type II Error

A false negative error occurring when a researcher fails to reject a false null hypothesis.

Effect Size

A quantitative measure of the magnitude of an experimental effect or relationship, indicating the degree of impact rather than just whether an impact exists.

Statistical Significance vs. Scientific Importance Matrix

A framework (Rosenthal et al., 2000) showing that combining a small p-value ($p < 0.05$) with a small effect size mistakes statistical significance for scientific importance, while a large p-value ($p > 0.05$) with a large effect size represents a failure to detect a scientifically important effect.

Publication Bias

The tendency for journals to publish papers with statistically significant results ($p < 0.05$) over non-significant ones, evidenced by distribution spikes in z-values just beyond $\pm 1.96$.

PLOS ONE Editorial Policy

A publication model that evaluates submitted studies solely on scientific rigor rather than perceived novelty or statistical significance, accepting well-conducted studies with $p > 0.05$.

Science Communication (Sarah Chow)

The field that equips individuals to break down, critique, and interpret scientific information from sensationalized news sources, thereby increasing science literacy and demand for accurate reporting.

"Two-Sided" Journalism Fallacy

The media tendency to present two contrasting perspectives on science-based topics (e.g., climate change or COVID-19) as equally valid, even when an overwhelming scientific consensus exists.

Cherry-Picking Data

The selective reporting of specific data points or short-term variations (such as a single-year increase in Arctic sea ice) to support a false premise while disregarding the true long-term trend.