ALHT211 Quick Reference: NHMRC Levels of Evidence, Study Designs & Data Visualisation

NHMRC Levels of Evidence (Overview)

• Focus on Levels III & IV in the slide set; Level II involves higher-level evidence syntheses.
• $\text{Level II}$: A systematic review of Level II studies
• $\text{Level III-1}$: Pseudo-randomised controlled trial
• $\text{Level III-2}$: Comparative study with concurrent controls (non-randomised)
• $\text{Level III-3}$: Comparative study without concurrent controls (historical controls, interrupted time series without parallel control; two or more single-arm studies)
• $\text{Level IV}$: Case series with post-test or pre-test/post-test outcomes; cross-sectional studies; case series; diagnostic yield studies without a reference standard
• NHMRC hierarchy covers different question types (screening, prognosis, aetiology, diagnosis)

Are lower-level studies worth appraisal?

• YES – they can inform practice where higher-level evidence is not available; essential for a complete appraisal

Pseudo-Randomised Trials (III-1) & Comparative Studies (III-2, III-3)

• Level III-1: Pseudo-randomised controlled trial (e.g., alternate allocation)
• Level III-2: Comparative study with concurrent controls (non-randomised, experimental trial)
  • Includes cohort studies, case-control studies, and interrupted time series with a control group
• Level III-3: Comparative study without concurrent controls (historical controls; interrupted time series without parallel control; two or more single-arm studies)
• Important note: Allocation bias is a concern in pseudo-randomised designs

What to check for: Cohort & Case-Control Studies

• Are comparison groups well described/defined?
• Were exposure and outcome measures collected in the same way (preferably blinded) across groups?
• Were all plausible confounds identified?
• What was the follow-up period?
• If historical controls were used, what differences in confounds could affect results?

Small-n Studies & Alternatives to RCTs

• Case Series (Level IV): one participant group; no controls; prospective; hypothesis-generating
• Surveys: questionnaires/interviews; quantify attitudes, beliefs, demographics
• Single Subject Experimental Designs (SSED/SCED): N = 1; ABABAB design; rigorous structure; focus on a single dependent variable
• Multiple SSEDs enable meta-analysis; SCRIBE guidelines (2016) exist for reporting

Case Studies & Qualitative Visualisation

• Case studies: detailed observations; useful for complex or unusual conditions
• Qualitative visualisation: word clouds, packed bubbles, icons/themes, quotes with stats

Visualising Data & Excel Skills

• Create basic graphs, tables, charts; choose visuals appropriate to data
• Distinguish descriptive vs. inferential statistics; integrate qualitative data themes where relevant
• Key graph types:
  • Bar graphs: nominal/ordinal data; frequencies or percentages; axis shows values
  • Histogram: interval/ratio data; shows distribution; bin widths reflect category width
  • Pie charts: nominal/ordinal data; slices sum to 100%
  • Scatterplots: two continuous variables; assess relationship/correlation
  • Boxplots: show spread (min, Q1, median, Q3, max) and outliers; whiskers to non-outliers

Why not RCT? & Key takeaways

• Reasons for not using an RCT: unchangeable variables, ethical concerns, cost, design fit
• RCTs focus on change and populations; may not explain individual differences
• Absence of a randomised trial is not absence of evidence: non-RCT evidence can be informative

Quick reference: Common study designs

• Cross-sectional: single time point
• Cohort: exposure vs non-exposure; follow-up for outcome
• Case-control: outcome present vs absent; look back at exposure
• Randomised Controlled Trial (RCT): random allocation; controlled
• Case Series: part of Level IV

References (key sources)

• Hoffmann et al. (2024); Portney & Watkins (2015); Polgar & Thomas (2020); Dupépé et al. (2019); Smith et al. (Field Trials of Health Interventions)