PHAR INFO LEC - 3.3 Critical Appraisal of Evidence

Critical Appraisal

Process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context

Steps of Critical Appraisal

1. Decide how trustworthy a piece of research is (Validity)

2. Determine what the research is telling us (Results)

3. Weigh up how useful the research will be in your context (Relevance)

Critical Appraisal Checklists

• Structured tools to systematically evaluate the validity, results, and relevance of research papers

• Done after publication

Reporting Guidelines

• Tools that specify a minimum set of items required for a clear and transparent account of what was done and what was found in a research study

• Done before and during publication

• Ensures completeness of a study

Placebo Effect

A psychological phenomenon where a patient experiences an improvement in symptoms due to the belief that they are receiving treatment.

Hawthorne Effect

Alteration of people's behavior when they are aware they are being observed

Measurement Bias

• Occurs when data or information is not accurately recorded in a research study

• Stem from errors in data collection, inconsistent measurement tools, or subjective interpretation of data

Publication Bias

Tendency for researchers and editors to handle the reporting of experimental results that are positive (i.e., showing a significant finding) differently from results that are negative (i.e., supporting the null hypothesis) or inconclusive

Observer/Experimenter Bias

When the person conducting the research allows their expectations or beliefs to influence the results of the experiment

Reporting Bias

Researchers selectively report or omit information based on the outcome of the research or personal beliefs

Recall Bias

When participants in a research study may not remember previous events or experiences accurately or they may subconsciously alter their memories.

Selection Bias

When the method of selecting participants or groups for a study produces an outcome that is not representative of the total population

Performance Bias

Differences that occur due to knowledge of intervention allocation, either in the researcher or participant

Attrition Bias

Distortion in outcomes in a clinical trial when there are unequal losses of participants from different study groups

Confirmation Bias

Tendency to favor, seek out, interpret, and remember information in a way that confirms one’s pre-existing beliefs or hypotheses while giving disproportionately less consideration to alternative possibilities

Data-Dredging Bias

• Probing the data in unplanned ways

• In the absence of a study protocol, researchers make analytic choices which will produce a statistically significant result

Confounding

• From confundere, “to mix together”

• A distortion that modifies the association between exposure and outcome because a factor is independently associated with the exposure and the outcome

Confounder

Third variable that distorts the association of two others

✓ It must be a risk factor for the outcome

✓ It must be associated with the exposure

✓ It must not be an intermediate step of the causal pathway

Restriction

Selecting subjects with a uniform factor that is considered a confounder

Matching

One subject in the control group will be matched to another in the exposure group with the same confounding factor/s

Randomization

Randomize subjects so that the exposure and control group have balanced out all their confounding factors

Chance

• Random error

• Uncontrolled variation or divergence between an observed or measured value of the sample from that of the true population value

• Samples selected from the population may misrepresent the population because of chance

• Unlike bias, can affect the results in both directions