Notes on Scientific Method, Peer Review, and Theory vs Law

Observation and the Scientific Method

Observation is the first step in a scientific inquiry. You observe something, which leads to questions like how, why, or what’s going on.
From the question, you formulate a hypothesis that aims at answering that question and has a hoped-for result.
A good hypothesis should be able to be supported by data and often addresses more than one dependent variable and independent variable.

Independent variable (IV): the variable you manipulate or change in the experiment.
Dependent variable (DV): the outcome you measure.
Control variables: factors you keep the same to ensure that observed effects are due to the IV rather than other factors. Example: grandma’s regular recipe as a baseline in a cookie experiment.
The design goal is to compare a baseline (control) with a modified condition to see the effect of the change.

A controlled experiment requires careful documentation: data collection, descriptions, materials, and methods.
The control variable ensures that only the intended change (IV) influences the outcome.
Experimental procedure example: use grandma’s regular recipe (control), substitute a new type of butter (IV), and compare outcomes (DV).
Results should be described and interpreted, with attention to what the data actually show, not just the nice-looking outcome.
It is important to discuss possible further research or follow-up experiments.

Results are not just the observed outcome (e.g., cookies); they should be expanded with analysis, context, and potential explanations.
Peer review is a critical step in science: results are evaluated by independent experts before publication.
Process of peer review:
- Write up the work in a format appropriate for the target publication.
- Editor assigns reviewers (peers in the same field) who are experts in that area.
- Reviewers assess and may request minor or major changes, or reject.
- The editor and a small board decide acceptance, revision, or rejection.
The review process helps ensure the research is credible and worthy of sharing with the scientific community.
With the Internet, information can spread quickly; therefore, relying on credible sources and understanding peer review is essential.

Peer-reviewed journals (e.g., top-tier biology journals) have stringent review processes:
- Science, Trends in Evolution, JAMA (Journal of the American Medical Association), and others with heavy peer review.
Some outlets imply high credibility but are not peer-reviewed:
- Popular press sites like New York Times, Newsweek, and similar outlets may report on studies, but their interpretation may differ from the original research.
- Some sites (e.g., Live Science) are science aggregators and may not provide original peer-reviewed content.
It’s essential to check the source of the information and, if possible, go back to the original journal article.
Paywalls can hide the primary material, making due diligence even more important.
An example discussed: a claim that masks do not work appeared in a letter to the editor referencing a study; the original study and context must be checked to avoid misinterpretation.

Distinguish between sources:
- Primary peer-reviewed research versus secondary reports or letters to the editor.
- Confirm whether statements reflect the actual study or a misrepresentation.
When encountering sensational headlines, trace them back to the original research and its context.
Always consider the possibility of misinterpretation or bias in reporting.
Due diligence involves verifying the source, context, and relevance of cited studies.

When asking a question about preferences (e.g., which of two options is preferred), avoid bias by using a blind or double-blind approach.
Blind study: participants do not know which treatment they received, but the researcher does.
Double-blind study: neither participants nor researchers know which treatment is administered until after data are collected.
Why blind/double-blind? To minimize bias from expectations or affiliations that could influence data collection or interpretation.
Example: taste test with two products labeled (or unlabeled in a controlled setup) to reduce expectations.
Another example: a new drug being tested on patients where researchers and participants are unaware of group assignment to prevent placebo effects or observer bias.

Common misunderstandings of the word theory in everyday language vs science:
- In science, a theory explains a broad and well-supported set of phenomena and is supported by evidence from multiple sources.
- A hypothesis is a testable statement that can be evaluated by experiments.
- A fact is a discrete, verifiable piece of information.
- A law is a descriptive generalization about how some aspect of the natural world behaves under certain conditions, often with mathematical expressions.
Example distinctions:
- Theory: the theory of gravity explains why objects fall and how gravity operates across scales; it is supported by extensive evidence.
- Law: the law of gravity describes quantitative relationships (e.g., the equations governing motion) and remains fixed under specified conditions (e.g., idealized cases).
A caveat: some theories describe relationships that are best expressed with mathematical formulations, while laws provide precise quantitative rules.
The gravity discussion in the transcript uses a vacuum experiment to illustrate that the theory of gravity may operate differently when air resistance is removed, highlighting the interplay between theory and conditions under which observations are made.
Terminal velocity concept (illustrated by air resistance):
- In a fluid, a falling object eventually reaches a constant speed when drag force balances weight.
- A common formulation: $$m g = rac{1}{2}
  ho Cd A vt^2 \Rightarrow v_t = \