Lecture 4: Levels of Evidence
Levels of Evidence
Needed to minimize and control biases
Rank the validity of evidence, allowing the user to put confidence in the results
Ex. A RCT provides stronger evidence than a cohort or case-control study when testing a therapy
Medical researchers and practitioners rank evidence according to its quality.
These levels when pictured together form a pyramid with the highest quality evidence at the top of the pyramid as there is less of this available.
This pyramid is often referred to as the "Evidence Pyramid".
Potential Biases in Clinical Research
Recall Bias: Occurs when participants do not remember previous events or experiences accurately or omit details.
Impact: Recall bias is a problem in studies that use self-reporting, e.g., case- control studies and retrospective cohort studies
Selection Bias: It is an error in assigning subjects to a study group where some members are less likely to be included than others, i.e., non-randomized selection.
Impact: Potential bias in observational studies such as cohort, case-control, and crossectional studies
Investigator Bias: Refer to situations in which researchers convey to research participants in subtle or unconscious ways the results or patterns the researcher would prefer to see
Impact: Can impact qualitative research in general
Placebo Bias: Occurs when study participants associate a treatment with a specific response. It has a large root in a person’s expectations
Impact: Can impact clinical trials in general; It can negatively impact non- randomized studies.
Social Desirability Bias: A.k.a. response bias, happens when participants may not answer questions entirely or truthfully in order to please the experimenter or based on what they wish their response to be
Impact: Can impact qualitative research in general, but particularly impactful in observational studies such as cohort, case-control, and crossectional studies.
Length Time Bias: It is an “apparent increase in survival time” resulting from a screening test favorably detecting less aggressive forms of the disease early in the disease process (as compared to another group screened later in the disease course).
Impact: Normally associated with non-randomized study designs.
Allocation Bias: It is the systematic difference between patients in how they are assigned to experimental groups.
Impact: This bias can be avoided by following a random assignment of subjects or using a double-blind study design.
Surveillance Bias: Happens when one group of participants is followed more closely than the other group and can result in increased diagnoses in the group being more closely observed. a.k.a. detection bias.
Impact: A common bias in non-randomized studies
Hawthorne Effect: Happens when an individual’s performance improves on a task simply because they know they are participating in an experiment
Impact: Can impact clinical trials in general; It can negatively impact non- blinded studies
Classification Bias: A.k.a. measurement or information bias, results from improper, inadequate, or ambiguous recording of individual factors—either exposure or outcome variables.
Impact: Because perfect tools to gather data are uncommon, most studies are subject to a certain degree of misclassification
Confounding Bias: It is a spurious association made between the outcome and a factor that is not itself causally related to the outcome; It occurs if the factor is associated with a range of other characteristics that do increase the outcome risk.
Impact: Confounding may be present in any study design
The Evidence Pyramid
Although each level may contribute to the total body of knowledge, not all levels are equally useful for making patient care decisions.
As one progresses up the levels, the number of studies and, correspondingly the amount of available literature decreases, while at the same time their relevance to answering clinical questions increases.
Knowing which type of research study provides the strongest level of evidence for the question being asked is important in conducting an evidence-based search of the literature.
These models for categorizing studies are helpful in determining the level of evidence available for answering clinically related questions and serve as a basis for identifying the strength of the evidence as being:
Strong (level 1)
Moderate (levels 2),
Limited/weak (level 3),
Missing evidence related to areas of practice (levels 4 & 5)
Relationship Between the Type of Question and the Type of Study
Clinical evidence in general will be judged on the rigor of its methodology
The level of evidence is directly related to the type of question asked, e.g., those derived from issues of therapy/intervention, diagnosis, etiology, and prognosis.
The highest level of evidence associated with questions about therapy or prevention will come from systematic reviews of RCTs studies because the objective of these studies is to test interventions demonstrating cause-and-effect and to select treatments that improve the condition/disease and avoid adverse events.
However, the highest level of evidence associated with questions about prognosis will be from meta- analysis or systematic reviews of inception cohort studies because the objective of these studies is to estimate the future course of a patient’s disease over time and to anticipate likely complications.
Inception cohort studies are those in which the cohort of subjects are all initially free of the outcome of interest and are followed until the occurrence of either a major study end point (death or full recovery) or the end of the study.
For diagnostic questions, the levels of evidence are related to the accuracy of the diagnostic test in terms of its sensitivity and specificity.
Sensitivity refers to a test's ability to designate an individual with disease as positive. A highly sensitive test means that there are few false negative results, and thus fewer cases of disease are missed.
Specificity of a test is its ability to designate an individual who does not have a disease as negative. A highly specific test means that there are few false positive results.
For questions related to new diagnosis, the highest level is systematic review of Controlled Trials carried under the following conditions:
Subjects/participants received both the new diagnostic test and the currently accepted reference or gold standard test
A range of participants was included: those who are disease free, have moderate disease, and have severe disease - so that the new diagnostic test can be evaluated for a broad scope of conditions.
Examiners were not aware of which test was used or the disease status of the subjects.
Results (sensitivity and specificity) were compared to see if the new test was as accurate as the currently accepted reference or gold standard test.
This procedure is considered a controlled trial, but not a randomized controlled trial because subjects receive both the new diagnostic test and the gold standard test
The purpose of systematic reviews of controlled trials is to determine the degree to which a test is reliable and useful.
Types of Clinical Questions and the Study of Choice to Answer them
Type of Evidence Related to Levels of Evidence and Study Methodology
At all levels, having systematic review or meta-analysis provides stronger evidence than a single study. However, it may not always be possible, which may lead you into searching for other resources.
For example: Suppose that you need to identify the best strategy for managing an endodontic lesion (a treatment question).
Ideally, a meta-analysis or systematic review of RCTs would be available that synthetized the research on endodontic treatment being considered.
If these were not available, then the next best evidence would be from a well-conducted individual RCT
Now, suppose that the focus of the question was on long- term outcomes of treatment (a prognosis question)
In this case, the highest level of evidence would be provided by a systematic review of inception cohort studies
And again, if a systematic review was not available, the next highest level would be an individual inception cohort study, and so on down the hierarchy.
Levels of Evidence Provide a Continuum of Knowledge- Water Fluoridation Example
It’s important to recognize that evidence may be used in all of its forms.
Each of the primary research study designs contributes to a continuum of knowledge development and validation
A classic example of this continuum is the concept of water fluoridation and its relationship to mottled enamel and caries incidence
Continuum Of Knowledge Derived from Different Study Designs (From simpler to complex)
Research Bias Practice
Identify the research biases in the cases described below:
1. When conducting a job satisfaction survey among your employees at your private
practice, you notice that every survey responder answered favorably to questions
regarding their feelings and perception of their workplace and denied ever having any
negative feelings about the work environment. What potential research bias have you
ignored when designing your survey? Social Desirability bias
2. While conducting a research study at the CUSOD, you ask patients who do not have internet access to answer an online survey when they get home. What research bias is likely to affect your research? Classification Bias
3. Early findings on the supposed beneficial effect of hormone replacement therapy in cardiovascular disease were reversed when studies that adjusted for socioeconomic status or education were accounted for. What research bias affected this research?
Confounding Bias
4. You are researching the smoking rates among CUSOD employees as part of a smoking cessation program. You collect your data by watching the employees during their work breaks. You’re trying not to be noticed because if employees know you are observing them, this can affect your study results. What research bias are you trying to minimize?
Hawthorne Effect
5. Parents of children diagnosed with cancer may be more likely to recall infections earlier in the child’s life than parents of children without cancer, leading the research results to show an untrue association between childhood infection and cancer. What research bias has affected this study Recall Bias
6. You are a researcher working on a survey containing questions about a new oral
diagnosis method you have helped to develop. While explaining the research to the
participants, who also happen to be your patients, you subtly share that you really hope this new method works for everyone. What bias are you introducing to your research? Investigator Bias
7. You normally take a specific white, oval-shaped pill for headaches and have begun to associate that pill with pain relief. When participating in a research study, you receive a similar-looking placebo pill for a headache. Even so, you still report decreased pain. What research bias is associated with your response? Placebo Bias
8. Nine previous randomized trials for a specific blood pressure medication have
consistently shown no changes in patients' results in either control or treatment groups. However, a new trial where patients’ group assignment wasn’t concealed resulted in imbalances in baseline blood pressure between the treatment and control groups. What research bias is affecting this study? Allocation Bias
9. While designing a retrospective study on systemic health conditions affecting patients seen at the CUSOD during 2010 – 2023, Dr. Sanchez observed that obese patients (BMI ≥ 30) were more likely to undergo medical examinations, blood tests, and imaging studies than non-obese people. By identifying this issue, what research bias can she prevent in her own study? Surveillance Bias
10. In a cancer study, researchers overestimated the survival length of patients due to an imbalance in the recruitment of patients whose cases were detected early and are slowly progressing relative to patients with more aggressive, fast-growing tumors. What research bias was introduced to this study? Length Time Bias
11. Research conclusions based only on the study of white male volunteers cannot be generalized to the full population, including women and people of other races or
communities. What research bias is at play here? Selection Bias