research pt 1

What is the highest level of evidence in the evidence hierarchy?

Systematic Reviews (SR) — sit at the apex of the pyramid, representing filtered information synthesised from the highest-quality primary studies.

List the levels of the evidence pyramid from bottom to top.

1. Background Information / Expert Opinion 2. Case-Controlled Studies / Case Series / Reports 3. Cohort Studies 4. Randomised Controlled Trials (RCTs) 5. Critically-Appraised Individual Articles (Article Synopses) 6. Critically-Appraised Topics / Evidence Syntheses 7. Systematic Reviews

What is 'filtered' vs 'unfiltered' information in the evidence pyramid?

Filtered: Systematic reviews, critically-appraised topics, article synopses — evidence has been appraised for quality. Unfiltered: RCTs, cohort studies, case-controlled studies — primary evidence not yet appraised.

What database searches both filtered and unfiltered evidence simultaneously?

The TRIP Database (Turning Research Into Practice).

Define a Systematic Review (SR).

A study type in which all available published and unpublished RCTs are reviewed; those meeting inclusion criteria are synthesised and the findings of re-analysis are presented to provide a high level of evidence on the effectiveness of healthcare interventions.

What is a meta-analysis?

A type of SR in which data from studies meeting strict inclusion criteria are combined and statistical analysis is repeated. It consists of two steps: 1. Extracting the size (and precision) of treatment effect from each study 2. Pooling the data to obtain a single weighted average estimate across all studies.

What is the main advantage of pooling results in a meta-analysis?

Increased sample size, which increases statistical power — making it more likely to detect a true effect if one exists.

How does a Systematic Review differ from a Literature Review in terms of its research question?

SR: Focused on a single, specific question. Literature Review: Not necessarily focused on a single question; may describe a broad overview.

How does a Systematic Review differ from a Literature Review in terms of protocol?

SR: A peer-reviewed protocol/plan is included before the review begins. Literature Review: No protocol is included.

How does a Systematic Review differ from a Literature Review in terms of inclusion/exclusion criteria?

SR: Criteria are stated before the review is conducted. Literature Review: Criteria are not specified.

How does a Systematic Review differ from a Literature Review in terms of search strategy?

SR: Comprehensive search conducted in a systematic way. Literature Review: Strategy not explicitly stated.

How does a Systematic Review differ from a Literature Review in terms of results synthesis?

SR: Clear summaries based on high-quality evidence with comprehensive quality evaluation. Literature Review: Summary based on studies of unspecified quality; may be influenced by the reviewer's own theories and beliefs.

Name three key databases used to source systematic reviews in physiotherapy.

1. Cochrane Library (cochrane.org/cochrane-reviews) 2. PEDro — Physiotherapy Evidence Database (pedro.org.au) 3. Joanna Briggs Institute (JBI Connect)

Name the main electronic databases searched for physiotherapy trials in a systematic review.

MEDLINE, Embase, CINAHL, AMED, PsycINFO, Cochrane, PEDro.

What are the 3 overarching questions used in critical appraisal of any study?

1. Are the results valid? (What steps minimise bias?) 2. What are the results? (Confidence intervals, significance?) 3. Will the results help me care for my patients? (Reproducibility, availability, applicability?)

What are the 3 main validity questions when critically appraising a Systematic Review?

1. Was it clear which trials were to be reviewed? (search strategy, PICO, inclusion/exclusion criteria) 2. Were most relevant studies reviewed? (comprehensive search, no publication/language bias) 3. Was the quality of reviewed studies taken into account?

When appraising a SR, what should you look for to determine if 'it was clear which trials were to be reviewed'?

• A clearly defined scope prior to searching • Clear inclusion and exclusion criteria (types of studies, participants, interventions, outcome measures — primary and secondary) • Use of a structured framework such as PICO • A documented Cochrane search strategy

What is PICO and when is it used in a systematic review?

PICO is a framework for defining inclusion/exclusion criteria: P = Patient/Population I = Intervention C = Comparison O = Outcome Used to ensure the review question and trial selection are clearly defined.

What biases can lead to missing relevant studies in a systematic review?

• Publication bias — negative studies less likely to be published • Language bias — excluding non-English literature • Grey literature bias — missing unpublished reports/theses • Recency bias — review may be outdated as new trials are published

Why is recency of a systematic review important?

New RCTs are published continuously. An outdated SR may miss important trials, so it is important to supplement older reviews by locating recent RCTs independently.

Why is it important to assess the quality of included studies in a systematic review?

• Allows determination of whether biases could have affected trial results • Biased primary studies → misleading SRs • Even if a SR follows high-quality methods, the overall evidence quality remains poor if the building blocks (primary studies) are poor quality

When should quality assessment of included studies be planned in a systematic review?

Quality criteria should be planned a priori (at the protocol stage), before trials are included. Formal assessment occurs after trial inclusion and prior to data extraction, usually by 2 independent reviewers.

List four tools used to assess trial quality within a systematic review.

1. CONSORT checklist 2. Jadad scale (checklist) 3. Cochrane Risk of Bias 2 (RoB 2) 4. PEDro scale (quality score, used in physiotherapy)

What is considered the single most important quality item when assessing trial quality in a SR?

Allocation concealment — ensuring that the person enrolling participants cannot predict which group they will be assigned to. Lack of allocation concealment is a major source of bias.

What other quality item (besides allocation concealment) has been empirically shown to make a difference in SR quality assessment?

Blinding of participants and outcome assessors. Note: blinding of therapists is rarely achievable in physiotherapy trials.

Is quality of trial reporting the same as quality of the trial itself?

No. Quality of reporting ≠ quality of the trial. A well-conducted trial may be poorly reported, and vice versa. It is better to describe quality items than simply score them.

For a systematic review, what measures are used to report effect size for continuous outcomes?

• Mean Difference (MD) — absolute difference between group means in a trial • Standardised Mean Difference (SMD) — used when studies assess the same outcome but measure it in different ways

For a systematic review, what measures are used to report effect size for dichotomous outcomes?

• Absolute Risk Reduction (ARR) — size of effect • 95% CI of ARR — precision of effect • Number Needed to Treat (NNT)

How do you determine statistical significance of a result in a systematic review?

• p-value • 95% CI does not cross 0 (for mean differences) or does not cross 1 (for risk ratios)

What is a forest plot and when is it used?

A forest plot is a graphical display of results from a systematic review / meta-analysis. Each study is shown as a line (95% CI) with a box (effect size, sized by weight). The pooled estimate is shown as a diamond. It is equivalent to the 'tree plot' used for a single RCT, but shows multiple studies at once.

In a forest plot, what does the size of each study's box represent?

The weight assigned to that study in the meta-analysis. Larger boxes = greater weight = more information (larger sample, more events, lower variance).

How is weighting of studies determined in a meta-analysis?

Weight is proportional to the inverse of the variance. Studies with more participants, more events, and lower variance receive greater weight because they give more precise estimates closer to the 'true' effect.

What is heterogeneity (I²) in a forest plot?

I² measures the degree of statistical inconsistency between studies in a meta-analysis. High I² (e.g., >50%) suggests studies are measuring different effects and pooling may be inappropriate.

What 3 questions guide the applicability of a SR's results to your patients?

1. Can the results be applied to my patient care? 2. Were all clinically important outcomes considered? 3. Are the benefits worth the harms and costs?

Define prognosis in the clinical context.

Prognosis relates to the probability or risk of an individual developing an outcome over a specific period of time based on their clinical profile.

List three clinical uses of prognostic information.

1. Inform patients about the likely course of their condition 2. Guide decision-making on treatments 3. Select patients for future trials on new treatments

What framework is used to structure a prognostic clinical question?

POT: P = Patient O = Outcome T = Timeframe Example: In patients with acute low back pain (P), what is the probability of being pain-free (O) within 6 weeks (T)?

Give three examples of prognostic questions relevant to physiotherapy.

1. In patients with acute low back pain, what is the probability of being pain-free within 6 weeks? 2. In patients with chronic low back pain, what is the probability of recurrence within 1 year? 3. In patients with knee osteoarthritis, what is the prognostic value of body weight change on pain/function within 3 years?

Where can prognostic evidence be sourced?

• Clinical observation/experience — limited (no routine follow-up, small samples) • Clinical research: – Prospective and retrospective cohort studies – Clinical trials – Systematic reviews Note: Prognostic data are often buried inside other study types and can be difficult to find.

What reporting checklist is used for prognostic (observational) studies?

STROBE statement (Strengthening the Reporting of Observational studies in Epidemiology) — strobe-statement.org

What are the 3 main validity questions when critically appraising a Prognostic Study?

1. Was there representative sampling from a well-defined population? 2. Was there an inception cohort? 3. Was there complete or near-complete follow-up?

What does 'representative sampling from a well-defined population' mean in a prognostic study?

The study must: • Clearly identify the target population with explicit inclusion/exclusion criteria • Recruit participants in a representative way — ideally all patients or consecutive cases • The greater the proportion of eligible participants recruited, the more representative the sample

What are 'threats to validity' regarding representative sampling in a prognostic study?

• Failure to sample representatively — e.g., only recruiting volunteers (not consecutive cases) • Failure to sample from a well-defined population — absence of clear inclusion/exclusion criteria

What is an inception cohort?

A cohort in which all participants are enrolled at an early and uniform point in the course of the condition (i.e., at or near disease onset). This avoids the bias inherent in 'survivor cohorts'.

What is a 'survivor cohort' and why is it a problem in prognostic research?

A survivor cohort is a group of patients who have already had the condition for a long time. It introduces bias because: • Only those who survived/persisted are included • The prognosis of a chronic condition differs from an acute one • Early outcomes (including worst outcomes) may be missed

Why is an inception cohort considered less biased than a survivor cohort, and what is its limitation?

Less biased because participants haven't been pre-selected by surviving the condition. However, inception cohorts are difficult to use for chronic conditions — you may need to wait many years to observe relevant outcomes.

What counts as complete or near-complete follow-up in a prognostic study?

• — tolerable • > 15% loss to follow-up — concerning; likely to cause bias • Whether deaths count as loss to follow-up must be considered

Why does loss to follow-up cause bias in prognostic studies?

Those who drop out may systematically differ from those who remain (e.g., they may have recovered or deteriorated more). This distorts the estimated prognosis for the whole cohort.

How is loss to follow-up calculated in a retrospective prognostic study?

Loss to follow-up = proportion of all eligible participants for whom follow-up data were NOT available. Formula: (Enrolled − Followed up) / Enrolled × 100%

In prospective vs retrospective prognostic studies, where is loss to follow-up easiest to identify?

Loss to follow-up is easier to identify in prospective cohort studies because participants are tracked in real time. In retrospective studies it must be calculated from available records.

What types of outcomes are reported in prognostic studies and how should they be expressed?

• Dichotomous outcomes — expressed as risk of the event • Continuous outcomes — expressed as mean (or median) of the outcome All results must be expressed in terms of a specific timeframe to be clinically useful.

How is effect size calculated for a dichotomous prognostic outcome?

Risk = n of people with outcome / n of people in sample Example: 10 ACL ruptures in 87 patients = 10/87 = 11% risk

How is the 95% CI calculated for a dichotomous prognostic outcome?

95% CI ≈ risk ± 1/√(2N) Example: risk = 11%, N = 87 1/√174 = 1/13.2 ≈ 8% 95% CI = 11% ± 8% → (3% to 19%)

How is effect size calculated for a continuous prognostic outcome?

Effect size = mean of the outcome for the group Example: Mean Neck Disability Index = 22.1 (SD 21.7) in N=99 patients.

How is the 95% CI calculated for a continuous prognostic outcome?

95% CI ≈ mean ± 1.96 × SD / √n Example: 22.1 ± 1.96 × 21.7/√99 = 22.1 ± 4.3 → (17.9 to 26.3)

What is a survival curve and what does it show?

A survival curve (e.g., Kaplan-Meier) is a graph that shows how the risk of an event changes over time. The y-axis shows survival (or probability of not having the outcome); the x-axis shows time. It is commonly used to display results of prognostic studies.

What 2 questions guide the applicability of a prognostic study's results?

1. Is the study relevant to me and my patients? 2. What does the evidence say?

What is the difference between statistical significance and clinical significance?

Statistical significance: p Clinical significance: Whether the size of the effect is meaningful for the patient and clinician in practice. A statistically significant result may not be clinically important.

What is allocation concealment and why is it important?

Allocation concealment ensures the person enrolling participants cannot know in advance which treatment group the next participant will be assigned to. It prevents selection bias and is considered the most important quality item in trial appraisal.

What is publication bias and why does it matter for systematic reviews?

Publication bias is the tendency for studies with positive or statistically significant results to be published more than those with null or negative results. This can cause a SR to overestimate the effectiveness of an intervention.

What is 'grey literature' and why should it be searched in a systematic review?

Grey literature = unpublished research (theses, conference abstracts, trial registries, government reports). It is searched to reduce publication bias and ensure the SR captures the full body of evidence, including studies with negative results.