Scientific Communication, Peer Review, and Scientific Expertise

Experimental Design and Scientific Method Foundations

Distinguishing Scientific vs. Non-Scientific Questions:
- Scientific methods evaluate empirical, testable hypotheses regarding observable phenomena in the natural world.
- An example of a question that can be answered using scientific methods is: How does caffeine affect the human body?
- Examples of non-scientific questions include: What is the meaning of life?, Does God exist?, and What is the best way to make new friends? These inquiries fall under philosophy, theology, ethics, or personal preference, and cannot be tested or falsified through empirical observation and experimentation.
Control Group Fundamentals:
- A control group is defined as the group of individuals or subjects in a scientific study that serves as a baseline comparison for the test (experimental) group.
- While the test group receives the specific treatment or variable being evaluated, the control group is kept under identical baseline conditions without receiving the experimental treatment.
Plant Germination and Growth Experiment Case Study:
- Experimental Design Setup: Two trays (Tray A and Tray B) were prepared with identical soil, and exactly $10$ seeds were planted in each tray.
- Environmental Conditions Applied:
- Tray A: Maintained at a temperature of $32^\circ\text{C}$ and supplied with $2$ cups of water.
- Tray B: Maintained at a temperature of $15^\circ\text{C}$ and supplied with $1$ cup of water.
- Quantitative Results at Day $6$ :
- Tray A produced $1$ growing plant out of $10$ seeds.
- Tray B produced $8$ growing plants out of $10$ seeds.
- Experimental Evaluation: It is impossible to conclude whether the plants grow better due to the lower temperature ( $15^\circ\text{C}$ versus $32^\circ\text{C}$ ) or due to the lower water volume ( $1$ cup versus $2$ cups).
- Methodological Flaw: The experiment altered two independent variables simultaneously (temperature and water volume), introducing confounding variables.
- Controlled Improvement: To establish valid causal relationships, experimental designs must isolate variables by altering only one variable at a time (e.g., testing different temperatures while holding water volume strictly constant across all trays).
Influenza Vaccine Testing Case Study in Mice:
- Experimental Design Setup: A scientist evaluated a novel influenza vaccine using two groups of mice. Exactly $10$ mice received the active vaccine, while $10$ mice received an inactive placebo. Subsequently, all $20$ mice were inoculated with the influenza virus.
- Experimental Improvement Strategy: To make the experimental results significantly more convincing and statistically robust, the sample size must be expanded by increasing the total number of mice tested per group.
- Invalid Modifications: Giving the mice different types of viruses or administering different types of vaccines simultaneously would introduce additional uncontrolled variables rather than improving the statistical validity of the core hypothesis.
Definition and Characterization of a Scientific Theory:
- A scientific theory is defined as an expansive, highly verified explanation of natural phenomena that is supported by an extensive body of empirical evidence gathered through repeated testing and observation.
- A scientific theory is distinct from a tentative, untested hypothesis or guess, and it is entirely distinct from pseudoscience.

Modes of Scientific Communication: Primary vs. Secondary Sources

Flowchart illustrating the scientific process across exploration, testing, outcomes, and community feedback

Overview of Communication Channels:
- Scientific knowledge is communicated across two main channels: primary sources for communication within the expert community, and secondary sources for communication to the broader public.
- Scientific claims require evaluation, testing, and community analysis by domain experts before they achieve credibility and integration into the broader body of scientific knowledge.
Primary Sources of Information:
- Definition: Primary sources are original documents that provide an unabridged, direct description of original empirical research written by the investigating scientists themselves.
- Essential Structural Contents: Primary sources contain complete accounts of research questions, experimental design, exact data collection methods, statistical analyses, raw data, and direct conclusions drawn by the investigators.
- Major Types of Primary Sources:
- Peer-Reviewed Scientific Journal Articles: Published papers documenting original research.
- Conference Papers and Abstracts: Manuscripts, poster presentations, and formal abstracts presented at professional scientific conferences.
- Doctoral Dissertations (PhD Theses): Formal treatises documenting original doctoral research defense.
- Scientific Patents: Legal filings describing novel technological inventions and biological processes.
- Institutional Purpose: Primary communication enables scientists to inform peers within their specific discipline of new findings, allowing independent experts to critically evaluate experimental design, analysis, and conclusions.
- Credibility Criterion: Scientific findings cannot establish formal credibility within the scientific community without being published in primary source outlets.

Front cover of Nature journal highlighting primary research publication

Key Primary Journal Examples and Empirical Studies:
- Nature (The International Weekly Journal of Science, Vol. $536$ , No. $7617$ , $25$ \,August\, $2016$ ) serves as a prominent primary source journal. A cover feature includes primary findings such as "Near Horizon: A warm terrestrial planet in orbit around Proxima Centauri, closest star to the Sun" (pages $408$ and $437$ ).
- Direct Field Research Example: Empirical documentation of semi-aquatic Anolis lizards by researcher Lindsey Swierk demonstrates specialized underwater rebreathing adaptations, where an air bubble maintained on the snout allows prolonged submergence.

Anolis lizard submerged underwater demonstrating an air bubble rebreathing mechanism

Secondary Sources of Information:
- Definition: Secondary sources are articles, reports, or media that analyze, interpret, summarize, or comment upon original research conducted by primary scientists.
- Major Outlets of Secondary Sources:
- News Organizations: Daily journalism, science reporting, and broadcasting networks.
- Opinion Pieces: Editorials, commentary columns, and analytical essays.
- Reference Works: Encyclopedias, educational portals, and general reference guides.
- Government Policy Documents: Legislative summaries, regulatory guidelines, and public health directives.
- Institutional Press Releases: Summaries issued by university communications offices and research facilities.
- Private Groups and Individuals: Publications by advocacy organizations, non-profits, and independent commentators.
- Public Function: Secondary sources act as the primary bridge conveying scientific developments to the general public by translating complex technical findings.
- Organizational Motivations and Reliability: Secondary source entities operate under widely varying motivations, including public education, product sales, behavioral modification, or policy advocacy. High reliability in a secondary source depends on whether its reporting accurately reflects the underlying scientific consensus.

The Peer Review Process and Pre-Print Repositories

Diagram detailing the sequential steps of the peer review process

Comprehensive Steps of the Peer Review Process:
- Step 1: Scientific Investigation and Manuscript Preparation — Scientists execute research studies, analyze empirical data, write a scientific manuscript detailing their findings, and submit it to an academic journal.
- Step 2: Editorial Intake — The journal editor receives the manuscript, conducts an initial assessment, and routes the manuscript to multiple external peer reviewers.
- Step 3: Peer Review Evaluation — The manuscript is assigned to between $3$ and $5$ active scientists who possess expert qualifications in that precise sub-discipline. Reviewers read the manuscript, evaluate experimental validity and analytical rigor, check for methodological errors or biases, and submit formal recommendations and feedback to the editor.
- Step 4: Editorial Determination and Workflow:
- Acceptance and Publication: If the manuscript satisfies all editorial and scientific standards, it is published in the journal.
- Revision and Resubmission: If reviewers identify flaws or missing data, the editor forwards reviewer comments to the authors. The scientists may revise their manuscript, execute supplementary experiments, and resubmit the paper for additional review.
- Rejection: If the manuscript fails to meet fundamental scientific standards or contains critical errors, the editor rejects the paper outright.
Academic Integrity, Limitations, and Retractions:
- Dependency on Integrity: The peer review process assumes academic honesty. Reviewers do not re-run experiments and cannot reliably catch intentional data fabrication or deliberate falsification prior to publication.
- Replication as Quality Control: Bad data or fraudulent claims are typically uncovered post-publication when independent research groups attempt to replicate the published experiments and fail.
- Corrections and Retractions: When critical errors or non-replicable findings are discovered post-publication, journals officially issue formal corrections (errata) or complete retractions of the published paper.

The bioRxiv pre-print server website homepage for biological research

Non-Peer Reviewed Dissemination Outlets:
- Conference Presentations: Abstracts and poster presentations submitted to scientific conferences permit rapid sharing of preliminary research based on basic administrative acceptance.
- Online Pre-print Repositories: Digital open-access archives, such as bioRxiv (hosted by Cold Spring Harbor Laboratory and supported by the Chan Zuckerberg Initiative), allow researchers to upload un-reviewed manuscripts.
- Subject Areas Hosted on bioRxiv: Host categories encompass Animal Behavior and Cognition, Biochemistry, Bioengineering, Bioinformatics, Biophysics, Cancer Biology, Cell Biology, Clinical Trials, Developmental Biology, Ecology, Epidemiology, Evolutionary Biology, Genetics, Genomics, Immunology, Microbiology, Molecular Biology, Neuroscience, Paleontology, Pathology, Pharmacology and Toxicology, Physiology, Plant Biology, Scientific Communication and Education, Synthetic Biology, Systems Biology, and Zoology.
- Utility of Non-Peer Reviewed Publishing:
- Solicit Early Feedback: Researchers upload pre-prints to obtain constructive feedback from the global scientific community before formal submission to a peer-reviewed journal.
- Emergency Public Health Response: Critical during rapid emergency events—such as the $COVID-19$ ( $SARS-CoV-2$ ) pandemic—where immediate dissemination of epidemiological, viral, and therapeutic data is necessary to advance public health long before standard peer-review timelines complete.

Scientific Expertise and Sub-Disciplines

Photograph of chemist and crystallographer Rosalind Franklin at a microscope

Defining Scientific Expertise:
- Scientific experts are individuals who have specialized in a specific subject area over many years, acquiring an authoritative comprehension of that domain.
- Necessary Qualifications and Indicators of Expertise:
- Advanced academic degrees, such as a Doctor of Philosophy ( $PhD$ ) or Doctor of Medicine ( $MD$ ), in the specific field under evaluation.
- Formal academic or research appointments at accredited research universities, national laboratories, or major scientific organizations (e.g., $NASA$ ).
- An extensive history of publications in peer-reviewed scientific journals.
- A documented track record of making significant contributions to their scientific discipline.
- Operational Capabilities of Experts:
- High likelihood of maintaining reliable, up-to-date scientific information.
- Specialized capability to comprehend complex, highly technical subject matter.
- Enhanced ability to accurately interpret complex empirical data.
- Proficiency in detecting subtle methodological errors, logical flaws, or systematic biases in research.
Scientific Career Training Path:
- Following completion of a Bachelor of Science ( $B.S.$ ) degree at a college or university, an individual training to become a scientist conducts immersive research in the laboratory of an established scientist over many years.
- Training within a specific laboratory develops deep, specialized expertise localized to a distinct discipline and sub-discipline.
Key Historical and Contemporary Case Studies of Scientific Expertise:
- Rosalind Franklin: Chemist and X-ray crystallographer whose research using X-ray diffraction techniques provided the critical structural photographs of biological molecules (including Photo 51) that revealed the helical structure of $DNA$ , making her a essential co-discoverer of $DNA$ structure alongside structural modelers.
- Neil deGrasse Tyson: Renowned astrophysicist, author, and science communicator who has authored books on cosmology and astrobiology. While highly trained in astrophysics and astronomy, his domain-specific technical expertise is astronomy, meaning his commentary does not carry expert authority in distinct disciplines such as evolutionary biology or human epidemiology.

Photograph of astrophysicist Neil deGrasse Tyson

Major Disciplines vs. Sub-Disciplines in Science:
- A scientific discipline represents an overarching domain of study (e.g., Biology, defined as the study of life).
- Sub-disciplines represent specialized domains within a broader discipline. Biology includes numerous specialized sub-disciplines:
- Anatomy: Study of physical structure of living organisms.
- Biochemistry: Study of chemical processes within living organisms.
- Biotechnology: Utilization of biological systems for industrial/technological applications.
- Botany: Study of plant biology.
- Cell Biology: Study of cellular structure, mechanisms, and functions.
- Developmental Biology: Study of growth and developmental processes of organisms.
- Ecology: Study of interactions among organisms and their environments.
- Environmental Biology: Study of biological systems within environmental contexts.
- Epidemiology: Study of the patterns, distribution, and determinants of health and disease states in populations.
- Evolutionary Biology: Study of evolutionary processes and genetic lineage adaptations.
- Genetics: Study of heredity and transmission of traits across generations.
- Immunology: Study of immune systems and host defense mechanisms against pathogens.
- Marine Biology: Study of organisms inhabiting ocean ecosystems.
- Microbiology: Study of microscopic organisms, including viruses, bacteria, and fungi.
- Neurobiology: Study of nervous system architecture and function.
- Paleontology: Study of prehistoric life through the fossil record.
- Pathology: Study of disease mechanisms, causes, and structural effects.
- Physiology: Study of physical and chemical functions in living systems.
- Zoology: Study of animal biology.
Specificity of Domain Expertise and Critical Red Flags:
- Scientific expertise is highly specific. A scientist trained in one sub-discipline (e.g., a botanist or ecologist) possesses general background knowledge in biology but lacks domain expertise in human clinical genetics or medicine.
- Red Flag Warning: Public commentary by scientists pontificating on subjects outside their precise domain of technical expertise (e.g., a physicist offering authoritative opinions on biological evolution, or non-epidemiologists offering expert mandates on pandemic management) represents a significant indicator of non-expert commentary.

Scientific Consensus and Secondary Source Evaluation

The Principle of Scientific Consensus:
- Scientific consensus is defined as the collective position, agreement, and judgment of the overwhelming majority of domain-specific expert scientists in a field, based on the systematic evaluation of all available empirical evidence.
- The Single-Expert Fallacy: A single scientist or expert can be incorrect, biased, or flawed in their interpretations. Reliable scientific conclusions depend on the broad consensus of the expert community rather than isolated individual claims.
Vaccine Evaluation Expert Assessment Scenario:
- Evaluative Scenario: Selecting the appropriate expert to analyze a scientific study on the effectiveness of a novel influenza vaccine.
- Candidate 1: An engineer who designs instruments to measure blood chemicals (lacks specialized knowledge in viral immunity).
- Candidate 2: A biologist who studies the impact of invasive species on ecosystems (specializes in ecology, not mammalian immunology).
- Candidate 3: A journalist who has read extensively about vaccines (lacks formal scientific research training and empirical expertise).
- Candidate 4: An immunologist who studies how the human immune system responds to viral pathogens.
- Analytical Selection: The immunologist is the only qualified domain expert capable of evaluating the study's immunology methods and vaccine efficacy conclusions.
Critical Evaluation of Secondary Information Sources:
- Web domain extensions (such as .org), professional layout designs, and official-sounding organization titles do not guarantee scientific reliability.
- Comparative Case Study of Health Organizations:

Homepage screenshot of the American Academy of Pediatrics website

- American Academy of Pediatrics ( $AAP$ , website: `https://www.aap.org/`):
  - The primary, major professional organization representing over  $67,000$  pediatricians across North America.
  - Publishes major peer-reviewed medical journals, establishes evidence-based clinical guidelines, and represents the established scientific consensus on child healthcare and epidemiology.

Homepage screenshot of the American College of Pediatricians website

- American College of Pediatricians ( $ACPeds$ , website: `https://acpeds.org/`):
  - A small, politically conservative advocacy group founded in  $2002$  by a small fraction of dissenting physicians.
  - Promotes socially conservative policy positions that frequently contradict established medical consensus and peer-reviewed pediatric literature.

Evaluation Criteria: Determining secondary source reliability requires scrutinizing membership scale, institutional background, funding sources, and alignment with peer-reviewed scientific consensus rather than relying on surface-level professional branding.