EBP and critical appraisal

What is Evidence-Based Care (EBP)?

Generating, evaluating and using the best evidence about the way to do the most good to people for the least harm. Integrates best research evidence with clinical expertise and applies it to individual patient needs.

What are the 3 pillars of Evidence-Based Physiotherapy?

1. High quality clinical research 2. Professional knowledge (clinical expertise) 3. Patient values & preferences

What are the 5 steps of EBP?

1. Ask – frame clinical question 2. Search – systematically search for best evidence 3. Critically Appraise – evaluate quality of evidence 4. Implement – apply results to patient 5. Evaluate – assess effectiveness of intervention

What 4 types of questions do most clinical queries concern?

1. Effects of intervention 2. Patients' experiences 3. Prognosis (course of condition) 4. Accuracy of diagnostic tests

What study design best answers an intervention/therapy question?

Randomised Controlled Trial (RCT)

What study design best answers a prognosis question?

Follow-up or inception cohort study

What study design best answers a diagnosis question?

Random or consecutive sample with a Gold Standard comparison

What study design best answers a frequency/prevalence question?

Random or consecutive sample

What study design best answers a phenomenological question (what is the experience)?

Qualitative research / observation

List the levels of evidence from lowest to highest (evidence pyramid).

1. Background information / Expert opinion 2. Case-controlled studies, Case series/reports 3. Cohort studies 4. Randomised Controlled Trials (RCTs) 5. Critically-appraised individual articles (article synopses) 6. Critically-appraised topics (evidence syntheses) 7. Systematic reviews

What is 'filtered' information in the evidence hierarchy?

Higher-level sources where someone has already appraised the evidence: systematic reviews, evidence syntheses, and article synopses.

What is 'unfiltered' information in the evidence hierarchy?

Primary sources not yet appraised: RCTs, cohort studies, case-controlled studies.

What are the 3 dimensions of evidence quality?

1. Strength (levels of evidence, risk of bias, statistical precision) 2. Effect size 3. Relevance

What are the 3 key questions to ask when critically appraising any study?

1. Are the results valid? (bias minimised?) 2. What are the results? (CIs, p-values, effect size) 3. Will the results help my patients? (reproducibility, availability, applicability)

What is internal validity in a study?

How well the study design controls confounding variables so the outcome reflects only the intervention being tested.

What is external validity in a study?

The extent to which results can be generalised from the study sample to a larger/different population.

What are the 4 main biases affecting internal validity?

1. Selection bias 2. Performance/intervention bias 3. Detection/measurement bias 4. Attrition bias

What quality assessment method addresses selection bias?

Allocation concealment

What quality assessment method addresses performance bias?

Blinding of investigator/subject

What quality assessment method addresses detection bias?

Blinding of outcome assessment

What quality assessment method addresses attrition bias?

Intention-to-treat (ITT) analysis

Define selection bias and give 3 subtypes.

Systematic error in who is enrolled. Subtypes: 1. Volunteer/referral bias – volunteers favour treatment group 2. Seasonal bias – timing of selection influences outcome (either direction) 3. Attention bias (Hawthorne effect) – awareness of being observed improves outcomes, favouring treatment

What is the Hawthorne effect?

Participants change their behaviour because they know they are being observed/receiving attention, biasing results in favour of the treatment group.

What is allocation concealment?

Ensuring the person enrolling participants cannot predict which group they will be assigned to, preventing selection bias in RCTs.

Define performance/intervention bias. Give examples.

Systematic differences in care other than the intervention of interest. Examples: - Contamination (control inadvertently receives treatment) - Co-intervention (participant receives another treatment outside the study) - Timing issues (insufficient time for effect / maturation in children) - Different therapists or treatment sites

Define detection/measurement bias. Give examples.

Systematic error in how outcomes are measured. Examples: - Too few outcomes (misses important effects) or too many (Type 1 error) - Lack of blinded/independent outcome assessment - Recall bias in self-report tools

What is a Type 1 error in the context of measurement bias?

False positive – concluding there is an effect when there isn't one. Risk increases when too many outcomes are measured with multiple statistical tests.

Define attrition bias.

Bias caused by drop-outs; if participants who leave differ systematically from those who stay, group comparability breaks down and results are skewed.

What is intention-to-treat (ITT) analysis?

Analysing all randomised participants in the group they were originally assigned to, regardless of whether they completed treatment, crossed over, or withdrew. Preserves the benefits of randomisation.

Name 3 critical appraisal tools used in physiotherapy EBP.

1. CASP – Critical Appraisal Skills Program 2. PEDro Scale (Physiotherapy Evidence Database) 3. CEBM tools – Centre for Evidence-Based Medicine, Oxford

What does PEDro stand for and what does it assess?

Physiotherapy Evidence Database. The PEDro scale rates RCT methodological quality out of 10 based on items like randomisation, blinding, and ITT analysis.

What are the key features of an RCT?

• Clearly identified patient population - Random allocation to 2 or more groups - One group receives intervention; other receives control (no treatment or standard care) - Allows group comparison - Reduces confounding variables

What are the 2 main disadvantages of RCTs?

1. Expensive (often require multiple sites and large samples) 2. Ethical concerns about withholding potentially beneficial treatment from the control group

What are the 3 main validity questions when appraising an RCT?

1. Were intervention and control groups comparable? 2. Was there complete or near-complete follow-up? 3. Was there blinding to allocation of patients and assessors?

How is group comparability achieved in an RCT?

• True random allocation (unpredictable by clinician) - Allocation concealment - Verified by baseline characteristics table (expect P > 0.05) - Maintained by complete follow-up and intention-to-treat analysis

Where can you find evidence of randomisation in a paper?

Title, abstract, or methods section. Look for description of method e.g. 'numbered, sealed, opaque envelopes prepared from random number tables'.

What dropout rate indicates potential attrition bias in an RCT?

More than 15% overall loss, OR a smaller total loss if the dropout rate is substantially greater in one group than the other.

What is the CONSORT flow diagram?

A standardised diagram showing participant flow through all stages of an RCT: enrolment, allocation, follow-up, and analysis. Makes it easy to track drop-outs and protocol violations.

What is blinding of participants in an RCT?

Participants do not know which group (intervention or control) they have been allocated to, preventing placebo effect bias.

What is a sham control group?

A control group that receives a fake/inactive version of the treatment so participants cannot distinguish it from the real intervention – enables true blinding.

What is blinding of assessors in an RCT?

The person measuring outcomes does not know which group the participant is in at the time of measurement, protecting against detection/measurement bias.

What is the placebo effect?

When patients benefit from an intervention that has no direct physiological effect, purely because they believe they are receiving treatment.

What is effect size in a continuous outcome RCT?

The difference in means between the treatment and control groups (mean difference).

What does a 95% Confidence Interval (CI) represent?

The range within which we are 95% certain the true population effect lies. Narrower CI = greater precision.

What is the formula for 95% CI for continuous outcomes?

95% CI = mean difference +/- (3 x SD_av) / sqrt(n_av) Equivalently: mean difference +/- 3 x SE_av

What p-value threshold indicates statistical significance?

p < 0.05

What is a dichotomous outcome?

An outcome that can only take one of two values, e.g. dead/alive, injured/not injured, satisfied/not satisfied.

What is Absolute Risk Reduction (ARR)?

The difference in event rate between control and intervention groups. ARR = risk_control - risk_intervention Also called risk difference.

How do you calculate 95% CI for ARR?

95% CI = ARR +/- (1 / sqrt(N_av)) x 100 where N_av = average group size.

What is Number Needed to Treat (NNT)?

Number of patients that need to receive the intervention to prevent one additional bad outcome. NNT = 1/ARR (or 100/ARR%)

What is Relative Risk Reduction (RRR)?

The proportion of baseline risk removed by the treatment. RRR = ARR / risk_control Useful for comparing studies but deceptive for clinical decision-making – prefer ARR and NNT.

What is relative risk (RR)?

Ratio of event risk in the intervention group to event risk in the control group. RR less than 1: intervention reduces risk RR greater than 1: intervention increases risk

What is an odds ratio (OR)?

Ratio of the odds of an event in the intervention group vs. control group. Used in meta-analyses where ARR/NNT are not well suited.

Why does baseline risk matter when interpreting NNT?

An intervention targeting a high-risk event yields a smaller NNT (more clinically worthwhile) than the same relative reduction applied to a rare event. The absolute benefit depends on the untreated baseline risk.

What is Minimal Clinically Important Difference (MCID)?

The smallest treatment effect that patients and clinicians consider worthwhile. Informed by patient perceptions of benefit and cost. Determines whether a statistically significant result is also clinically meaningful.

What factors affect whether RCT results can be applied to your patient?

• Are study participants similar to your patient? - Were interventions applied appropriately and in a theoretically reasonable way? - Are outcomes relevant and meaningful to patients? - Recency of the study - Similar clinical setting - Conclusions match the results - Limitations are reported

What is the difference between statistical significance and clinical significance?

Statistical significance (p < 0.05) means the result is unlikely due to chance. Clinical significance means the effect is large enough to matter to the patient/clinician. A result can be statistically significant but clinically trivial.

What questions help assess whether an intervention's benefits outweigh harms and costs?

• Is it worth the time, effort, and discomfort? - Does it do more good than harm? - Does the effect size exceed the MCID? - Does it suggest policy change? - Can you defend the choice in a legal context?

What does it mean to 'assess the relevance' of an RCT?

Asking whether: participants are similar to your patient; interventions were applied appropriately; outcomes matter to the patient; results are generalisable to your local population; the study is recent enough.