Notes on the Scientific Method: Observation, Reliability, and Pseudoscience

Scientific Method: Living Things, Evidence, and Public Trust

  • Life science = study of living organisms; uses the scientific method to study living systems.
  • The scientific method is a set of rules developed over decades that has helped answer questions about how the world works.
  • Scientists report their findings; the public evaluates and may call for changes based on observations.
  • The scientific process is not linear; relationships do not automatically imply causation. Correlation ≠ causation.

Non-linearity, Causation, and Interpreting Results

  • Relationships in science can be linear or non-linear; complexity makes causal inference difficult.
  • When two researchers study the same topic, they can obtain different results due to various factors (examples and biases discussed below).
  • The wine example: two subjects in a study on wine can yield different outcomes; highlights variability in human results and the influence of context.
  • Biases and design differences can drive divergent results:
    • Sampling bias: one group more biased in participant selection.
    • Health differences in participants (e.g., age ranges like 75–80) and volunteer bias.
    • Control group differences and whether the control group truly matches the experimental group.
    • Sponsorship or financial interests can influence results.
  • Family history (genetics) can influence results (e.g., hereditary heart disease).

Reliability, Replication, and Evidence Synthesis

  • If two studies have opposite conclusions, multiple factors could be at play.
  • Science seeks reliability through replication and accumulation of evidence.
  • If you have many studies showing a similar result (e.g., humans are warming the planet over the last ~${$10^{2}}$} years), a single opposing study is weighed against the larger body of evidence.
  • The repeatability of results strengthens scientific confidence; more studies confirming a result increases trust in that finding.
  • Example from the speaker’s field: conflicting studies on how bird size affects pitch, illustrating that not all questions have a single, settled truth yet.

The Iterative, Competitive Process of Science

  • Science is iterative and competitive: many studies ask the same questions, repeat experiments, and accumulate evidence.
  • Over time, this process can converge toward a scientific truth for certain questions.
  • If there is incomplete understanding or conflicting results, more studies are conducted to clarify.

Components of the Process of Science

  • Observation: careful watching to gain information.
    • Examples: hotter summers in August vs December; earlier sunsets in July vs March.
  • Hypothesis: a testable explanation for observed phenomena.
    • Hypotheses can arise from logical reasoning, experience, chance, intuition, or established theory.
    • Examples:
    • Hypothesis from experience: being cold and wet after swimming causes illness.
    • Literature examples: cuttlefish skin showing millions of colors without a power source, suggesting inspiration for non-powered display tech (hypothetical).
    • Hypotheses should be testable and supported by data; they should explain phenomena in a valid, measurable way.
    • The hypothesis process can blend creative and logical thinking.
  • Literature review: examine existing reports and findings before forming new tests.
  • Testability and falsifiability:
    • Testable hypothesis: can be supported or refuted by data.
    • Non-testable (non-falsifiable) hypothesis: cannot be measured or disproven (e.g., psychic energy explanations).
    • Distinguishing between testable and non-testable is crucial; science can lean on philosophy for non-testable questions.
  • Pseudoscience vs science:
    • Pseudoscience features: vague or babble-like language; lack of a coherent mechanism; extraordinary claims with insufficient evidence.
    • Science prizes evidence: receipts, data, and reproducible results.
    • Pseudoscience often lacks peer-reviewed validation and relies on non-testable claims.
  • Falsifiability and calcifiability:
    • A falsifiable (calcifiable) hypothesis can be disproven by evidence.
    • A non-falsifiable hypothesis cannot be proved wrong and is not scientifically testable.
  • Peer review:
    • Scientific claims are evaluated by peers in the same field before publication.
    • Peers check for flaws, repeat experiments if needed, and provide validation or critique.
  • Conspiracy claims vs evidence:
    • Scientific practice requires testability and verifiability (receipts/evidence).
    • Conspiratorial claims are not considered scientific without testable evidence and reproducibility.

Practical Implications for Evaluating Scientific Claims

  • When faced with conflicting results, seek additional studies and replication rather than accepting a single study.
  • Consider potential biases: sampling bias, control group differences, volunteer bias, sponsorship bias, and social biases.
  • Recognize that science is not about certainty in every case; it is about the weight of evidence built through repeated testing and convergence of results.
  • Acknowledge the role of philosophy in questions that are not readily testable by current methods (e.g., psychic energy).
  • Remember the maxim frequently invoked in science: show me the receipts. Evidence, reproducibility, and peer review are essential.

Real-World Takeaways and Examples Mentioned

  • The wine study controversy illustrates how different subject pools can yield different results.
  • The smoking example highlights long-term effects and timescales (roughly a decade) used to assess health outcomes.
  • The broader point that data, statistics, and the mean (average) help summarize effects across groups:
    • Mean is often used to describe central tendency: ar{x} = \frac{1}{n}\sum{i=1}^{n} xi
  • The discussion about larger birds and pitch demonstrates that not all scientific questions have a settled answer; more studies can resolve some questions while others remain open.
  • Cuttlefish color patterns as an example of how literature can inspire real-world technology and hypotheses (even if the example is hypothetical in this context).
  • The role of replication: multiple studies addressing the same question increase confidence in findings; a single contradictory study is not sufficient to overturn a broad consensus.
  • The overall message: science is an iterative, evidence-based, and public enterprise that advances by building reliability, addressing biases, and differentiating testable from non-testable questions.