Levels of Evidence Notes

Introduction to Levels of Evidence

• Levels of evidence are critical in evaluating the quality of research and understanding potential biases in studies.

• They provide a framework for determining the strength of the findings based on research design and methodology.

• Evidence-based practitioners utilise these levels to discern the best evidence available for their practice, ensuring that decisions are grounded in scientifically sound research rather than anecdotal evidence or personal opinions.

Understanding Bias

Definition of Bias:

• Bias refers to any effect or influence that prevents study conclusions from being accurate. It can manifest in various forms and can significantly distort research outcomes (Hoffmann et al., 2017).

• Bias can be recognised as systematic errors that occur during data collection, interpretation, or reporting of results.

Characteristics of Bias:

• Bias is typically consistent (not random) which makes it predictable in some instances, yet it often remains undetectable in practice.

• Bias can lead to either the overestimation or underestimation of effects in research conclusions, which can ultimately misinform practitioners and policy-makers about effective interventions.

Importance of Study Design

• The design of a study can significantly impact its risk of bias.

• Good research design is crucial in minimising biases that can arise throughout the study's execution.

• Well-designed studies, which adhere to strict methodological protocols, typically demonstrate lower risks of bias, yielding more reliable results for practitioners.

• Conversely, poorly designed studies may lead to misleading conclusions, as they might introduce confounding factors and increase variability in the results.

• Understanding the design is essential for interpreting the validity of the findings.

NHMRC Evidence Hierarchy

NHMRC Levels of Evidence (2009) Classification:

The NHMRC (National Health and Medical Research Council) classifies evidence levels ranging from I (highest quality, lowest risk of bias) to IV (lowest quality, highest risk of bias).

This hierarchy assists in guiding practitioners toward the most reliable research.

Level I: Systematic reviews of Level II studies, which synthesise the outcomes of multiple rigorous studies to provide comprehensive insights.

Level II: Randomised controlled trials with true randomisation, recognised as the gold standard in clinical research for minimising bias.

Levels III-1 to III-3: These levels include various controlled studies with less stringent randomisation or controls, with increasing risk of bias. They might include cohort studies or case-control studies that provide important epidemiological data.

Level IV: Case series with no control groups, representing the weakest category due to the high risk of bias associated with a lack of comparison.

Research Designs and Their Risks

• Higher levels of evidence correlate with lower numbers in the NHMRC hierarchy, indicating a stronger foundation for findings:

• Level I: This level is superior due to the combination of results from at least two eligible studies, thus providing broader evidence for clinical practice.

• Level II: Based on randomised controlled trials—considered the gold standard for establishing cause-and-effect relationships.

• Level III: This includes controlled trials without true randomisation, resulting in a higher risk of bias and less reliable conclusions.

• Level IV: Represents the weakest design due to the absence of a control group, making it challenging to draw robust conclusions.

Analysing Evidence Quality

Limitations of NHMRC Levels:

• While a high level of evidence indicates a more rigorous study, it does not always ensure that the study execution was thorough or devoid of bias control.

• Levels of evidence primarily focus on study design, yet they do not encapsulate all potential biases. Crucial factors that might influence the evidence quality include:

• Sampling issues (including attrition, which can lead to bias if those lost to follow-up differ from those who complete the study).

• Measurement errors that can skew data (e.g., improper tools or techniques used in data collection).

• Analysis methods that might inadvertently introduce bias if not conducted correctly.

• A study categorised with a lower NHMRC level could still exhibit robust quality if it is well conducted and mitigates biases effectively.