Lec 11 - limitations of the scientific method

What is the goal of the scientific method?

To test possible explanations (hypotheses) by designing experiments and collecting evidence to see if predictions agree with the results

Are scientific conclusions permanent?

No, conclusions are always provisional — new evidence can lead to revisions and updated understanding.

What can happen to a hypothesis during testing?

It can only be supported or rejected, not proven absolutely true.

What does “provisional conclusion” mean in science?

It means the conclusion is accepted for now but may change with new evidence or discoveries.

Limitation #1 — Scope of the Main Question

Science is limited to studying questions that can be observed, measured objectively, and tested repeatedly. It cannot address moral, ethical, or value-based questions, nor fully capture complex, interconnected phenomena that cannot be isolated for testing.

What kinds of questions fall outside the scope of the scientific method?

Moral, ethical, and value-based questions (like “right vs. wrong” or “good vs. bad”) because they cannot be measured or tested objectively.

Why might some complex phenomena be difficult to study scientifically?

Because they involve interconnected systems that are hard to isolate into simple, testable parts.
Example: Predicting the long-term effects of human-driven climate change on biodiversity, which involves many interacting variables.

Limitation #2 — Available Knowledge, Resources, and Technology

The quality of scientific investigation depends on the current state of knowledge, available resources, and technology. Outdated information, limited access to tools, or inadequate technology can restrict experimental design, accuracy, and feasibility of research questions.

: Give an example of how improved technology has expanded scientific research.

The development of wearable ECG and EEG devices allows continuous, portable monitoring for conditions like arrhythmia and epilepsy, whereas patients once had to be in a clinic for such recordings

.

Limitation #3 — Ethics and Practicality

Scientific studies are restricted by ethical standards and practical constraints that protect participants and the environment. Researchers must obtain informed consent, avoid harmful or invasive experiments, and respect privacy and safety regulations, even when these limits make certain studies difficult or impossible to perform.

How do ethical and practical concerns limit scientific research?

How do ethical and practical concerns limit scientific research?
A: Some studies cannot be performed because it’s unethical or impractical to control variables or expose participants to risk — for example, studying pregnancy or embryo development is strictly regulated.

Why are pregnant people often excluded from clinical trials?

Due to ethical concerns about unknown risks to the fetus, even though this exclusion limits data on treating infections and chronic diseases in pregnant individuals.

Limitation #4 — Confounders and Bias

Scientific results can be distorted by confounding variables—factors that affect both the treatment and the outcome—making it unclear what truly causes the effect. To minimize this, researchers use careful study design and controls for variables like age, sex/gender, and socio-economic status to ensure conclusions are accurate.

What is a confounder in scientific research?

A variable that influences both the exposure and the outcome, potentially creating a false association or hiding a real one.

Example: In a study on exercise and heart disease, age could be a confounder because it affects both how much someone exercises and their risk of heart disease.

How can researchers reduce the effects of confounders and bias?

By using good study design—such as randomization, control groups, and statistical adjustments—to isolate the true relationship between treatment and outcome.

What is bias in scientific research?

Bias occurs when systematic errors or personal influences distort study results, making them less trustworthy.
Example: A researcher expecting a drug to work may unintentionally interpret data in favor of the treatment.

What is sampling bias, and why is it a problem?

Sampling bias happens when participants chosen for a study don’t represent the larger population, which can lead to misleading conclusions.
Example: Studying a treatment only in young adults may give inaccurate results for older adults.

Limitation #5 — Sampling

The size and representativeness of a sample greatly affect the reliability of study results. Small sample sizes can lead to inaccurate estimates and wider confidence intervals, increasing the chance that results don’t reflect the true population mean.

How does sample size affect the reliability of scientific results?

Smaller samples produce less reliable results and wider confidence intervals, making it more likely that findings don’t capture the true population mean.

what is the main takeaway about sampling in research?

Small sample sizes increase random error and reduce confidence in results — for example, in 20 samples, some intervals may miss the true mean just by chance.

What factors determine how large a sample should be in a study?

Effect size, population variability, desired confidence and power, margin of error, study design, and available resources.- The smaller the effect you’re trying to detect, the larger the sample required to notice it reliably.

• smaller margin of error (tighter confidence interval) requires a larger sample.

Limitation #6 — Measurement

Measurement errors or biases can distort scientific results. These errors may arise from technical problems, calibration issues, or subjective interpretation of results. Even participants’ awareness of being measured can alter their behavior, leading to measurement bias and less accurate data.

What is measurement bias, and how can it affect results?

Measurement bias occurs when a technique or instrument systematically favors certain outcomes, leading to inaccurate results.
Example: Using inconsistent X-ray interpretations may cause errors in diagnosing viral pneumonia.

How can being observed change a participant’s behavior?

People may act differently when they know they’re being measured — a phenomenon seen in White Coat Syndrome, where patients’ blood pressure rises in clinical settings due to anxiety but remains normal elsewhere.

Limitation #7 — Human Error and Fraud

Scientific reliability can be compromised by human mistakes, such as data entry or analysis errors, and by intentional misconduct like data fabrication or plagiarism. Even with peer review, these problems may go unnoticed, reducing trust in research findings.

How can honest human error affect scientific research?

Simple mistakes in data collection, recording, or interpretation can distort findings or lead to incorrect conclusions.
Example: Mislabeling samples or entering numbers incorrectly during analysis.

What is scientific fraud, and why is it harmful?

Scientific fraud is the deliberate falsification, alteration, or copying of data to make results seem more successful than they are.

Example: A researcher fabricating experiment results or plagiarizing another scientist’s work undermines the credibility of all related research.

Limitation #8 — Reductionist Philosophy

Reductionism simplifies complex systems by studying their parts in isolation to find single causes. While effective for identifying specific mechanisms, it may overlook how multiple factors interact. In contrast, holism examines systems as a whole, focusing on relationships and context — often providing a more complete understanding of broad or interconnected phenomena.

What is the main limitation of reductionism in science?

It can oversimplify complex systems, missing how multiple factors or interactions contribute to outcomes.

Example: Studying a single hormone’s effect on mood without considering social or environmental influences.

How does holism differ from reductionism?

Holism considers the whole system and how its parts interact, making it better for studying broad or interconnected patterns — for example, understanding how lifestyle, genetics, and environment combine to influence disease.

summary slide of how we can do better to use the scintific metopd

Limitations of the Scientific Method

1⃣ Scope – can only test observable, measurable questions, not moral or value-based ones.
2⃣ Knowledge & Technology – outdated tools or data restrict accuracy.
3⃣ Ethics & Practicality – some studies are unsafe or unrealistic.
4⃣ Confounders & Bias – uncontrolled variables or researcher bias distort results.
5⃣ Sampling – small or unrepresentative samples reduce reliability.
6⃣ Measurement – errors, calibration issues, or behavior changes affect accuracy.
7⃣ Human Error & Fraud – mistakes or misconduct compromise validity.
8⃣ Reductionism – focusing on isolated parts can miss broader system interactions.