Synthesizing Evidence Notes

Introduction to Synthesizing Evidence

The lesson focuses on the crucial skill of finding and synthesizing data for evaluations, addressing common questions about data sources.
Many evaluation works rely on educated guesses; this lesson aims to inform those guesses by teaching how to find and use relevant data.
Unlike previous exercises where data was provided, you will now learn to locate and apply data to your studies.

The Importance of Data Synthesis

Data synthesis involves combining data from various sources to create parameters for models.
Often, directly applicable data is unavailable, requiring the use of related outcome data to inform study parameters.
The key challenge lies in determining the relevance of data to the research question.

Using Results from Other Studies

In many cases (over half), results from studies in similar areas can be used.
For example, in asthma studies, outcome rates for asthma can inform assumptions in your models.
A simplifying assumption can be made, such as assuming the hospitalization rate for asthma with an intervention will be similar or slightly better, with proper justification.

Synthesizing Data Elements

Sometimes, you'll need to combine elements from different studies, justifying their relevance to your research question.
The studies used must have some degree of generalizability, which can be assessed through their limitations statements.

Assessing Generalizability

Refer to the limitations section of peer-reviewed research to find discussions on generalizability.
Use these discussions to explain why the data is applicable to your study.
This approach is necessary when using interventions from other studies that aren't exactly the same as yours.
Be prepared to defend the use of these resources by addressing their limitations and strengths.
Simply citing a peer-reviewed source is insufficient justification.

Selecting Evidence

Prioritize newer studies over older ones when they present similar findings, ensuring the latest details are used.

Systematic Approach to Evidence Gathering

Adopt a systematic review or meta-analysis approach to find relevant evidence.
Define parameters for relevant evidence and determine appropriate sources for searching.
Set a reasonable time frame (e.g., the last 5-10 years) based on the amount of research in your area.
Emphasize a systematic approach rather than randomly selecting sources based on abstracts or titles.

Considering Location and Comparability

Prefer studies from the same country as your study to ensure comparability.
Be cautious when using studies from regions with significantly different healthcare systems or socioeconomic statuses, such as the Far East or Africa.
Ensure some degree of comparability in the populations being evaluated.

Addressing Bias and Conflicting Data

Actively look for bias in the data and gather data that both supports and contradicts your hypothesis.
Address conflicting data by providing reasons why certain studies may not align with your hypothesis (e.g., different selection criteria).
Acknowledging and explaining conflicting data strengthens your argument.

Critical Evaluation of Data

Critically assess data by questioning potential issues and whether the data makes sense.
Evaluate the methodology used in the study for reasonableness.
Check if the results align with expectations and prior studies.
If results differ, look for a rationale that explains the discrepancy.

Prioritizing Domestic Studies

Lean toward domestic studies (e.g., within the U.S.) unless there is a lack of available data.
Acknowledge the limitations when using international sources due to differences in study populations.
Addressing such limitations enhances the credibility of your study.

Limited Reviews for Specific Values

Conduct limited reviews when searching for specific values or alternatives necessary for your study.
For example, find the positivity rate for COVID after a second immunization for Hispanic populations aged 65 and above.

Borrowing Approaches from Other Studies

Consider how similar questions have been studied previously and potentially borrow those research approaches.
Use parameters from studies with successful approaches, citing appropriately.

Identifying Relevant Statistics and Measures

Look for statistics such as rates of cancer progression, hazard ratios, and mortality rates.
Examine evidence for side effects, which can be measured in decision trees or Markov models.
Gather information on how disease conditions were assessed for health-related quality of life measures, especially for cost-utility analysis.

Adjusting Costs for Time

When using cost data from other studies, adjust the costs to the current year of your study.
For example, adjust cost data from 2015 to 2022 dollars to account for inflation.
Failing to adjust costs for time can weaken the quality of your work and reduce the likelihood of publication.