Week 6 Notes – Evidence Synthesis & Evidence Tables

Introduction to Evidence Synthesis

  • Purpose & Value

    • Compiles findings from multiple primary studies to build a coherent picture of a phenomenon.

    • Detects overall trends, reveals consistencies / discrepancies, and directs future research agendas.

    • Strengthens external validity by moving beyond single‐study idiosyncrasies.

  • Typical Workflow (recap of previous weeks)

    • Formulate focused question → Systematic search & screening → Critical appraisal → Synthesis (this week’s focus) → Interpretation & reporting.

  • Two Core Approaches

    • Narrative (qualitative) synthesis – text-based integration of heterogeneous evidence.

    • Systematic (quantitative) synthesis / meta-analysis – statistical pooling of comparable effect sizes.

    • Choice depends on data homogeneity, research objective, and available expertise.

Narrative Synthesis

  • Definition: Qualitative strategy that describes and interprets findings across diverse designs, measures, and contexts.

  • When Preferred: High methodological diversity, scarce quantitative data, exploratory questions.

  • Core Techniques

    1. Thematic Analysis

    • Code recurring concepts, group them into overarching themes.

    • Example: Interventions for university anxiety → themes of coping strategies, social support, therapeutic modality effectiveness.

    1. Tabulation

    • Build matrices to juxtapose study characteristics & outcomes.

    • Enables side-by-side comparison (e.g., population, intervention type, outcome score).

    1. Textual Description

    • Rich, critical description of each study plus cross-study linkages.

    • Example: Contrast varying definitions of resilience and discuss how operationalisation affects reported outcomes.

  • Structuring a Narrative Review

    • Introduce scope → Present thematic clusters → Contrast findings → Identify gaps → Conclude with implications.

    • Integrate critical appraisal rather than listing studies uncritically.

  • Real-World Relevance: Often underpins practice guidelines when statistical pooling is impossible (e.g., early evidence on emerging therapies).

Quantitative Synthesis: Meta-Analysis

  • Goal: Produce a pooled effect size offering greater precision than any individual study.

  • Key Steps

    1. Effect-Size Calculation

    • Convert disparate results to a common metric (e.g., Standardised Mean Difference, d; Odds Ratio, OR).

    • Formula example: \text{SMD}=\frac{\bar X{\text{treat}}-\bar X{\text{control}}}{SD_{\text{pooled}}}.

    • CBT example: 5-point drop on 20-pt scale vs 0.8-pt drop on 5-pt scale → compute comparable d values.

    1. Statistical Integration

    • Choose fixed-effect (assumes common true effect) or random-effects (allows effect variance).

    • Weight each study (inverse of variance); compute pooled estimate & 95 % CI.

    1. Heterogeneity Assessment

    • Cochran’s Q and I^{2} statistics quantify dispersion.
      I^{2}=\frac{Q-df}{Q}\times100\% (values > 50 % ≈ substantial heterogeneity).

    • Address via sub-group, meta-regression, or sensitivity analyses (e.g., age groups, baseline severity).

  • Quality Safeguards

    • Ensure comprehensive search, consistent coding, duplicate extraction, publication-bias tests (funnel plot, Egger).

  • Ethical Implication: Mis-pooled biased studies can misguide treatment guidelines; transparency is obligatory.

Mixed-Methods Synthesis

  • Concept: Integrates statistical generalisability of quantitative data with the contextual richness of qualitative findings.

  • Advantages

    • Validates quantitative trends with participant narratives; illuminates mechanisms.

  • Typical Challenges & Mitigations

    • Data Integration Complexity

    • Design a priori plan (convergent, sequential, or embedded synthesis design).

    • Example: Merge RCT anxiety score reductions with interview-based perceptions of CBT usefulness.

    • Consistency in Interpretation

    • Use joint-display matrices to align qualitative themes with quantitative outcomes.

    • Ensure narratives support rather than contradict statistical trends (or explain divergence).

Developing Synthesis Tables

  • Purpose: Condense complex multi-study information into a clear, accessible snapshot.

  • Construction Principles

    1. Organise Information

    • Essential columns: Author/Year | Design | Sample (N, characteristics) | Intervention/Exposure | Outcome Measures | Effect Size / Key Findings | Quality Rating.

    • Example entry for CBT study: Jones et al., 2022 | RCT | N=120 undergraduates | 8-week group CBT | Beck Anxiety Inventory | d=-0.65 | Low risk of bias.

    1. Visual Presentation

    • Bold headers, uniform fonts, shading/colour for subgroup distinctions.

    • Keep row heights aligned; wrap text judiciously.

    1. Data Summarisation

    • Highlight repeating findings (e.g., majority show 10 %–20 % reduction).

    • Include footnotes for anomalies or methodological caveats.

  • Toolbox

    • Microsoft Excel / Google Sheets → Sorting, filtering, conditional formatting.

    • Reference-management plug-ins (e.g., Zotero → CSV export) accelerate population.

Critical Evaluation of Synthesised Evidence

  • Quality Assessment

    • CASP, Joanna Briggs, Cochrane Risk of Bias 2; record domains such as randomisation, blinding, attrition.

  • Applicability / External Validity

    • Compare study populations with target context; note cultural, age, or clinical severity boundaries.

  • Biases & Limitations

    • Publication bias (file-drawer effect) may inflate pooled effect; explore with funnel plots, trim-and-fill.

    • Language bias, selective outcome reporting, researcher allegiance.

  • Transparent Reporting: Follow PRISMA or ENTREQ checklists; supply supplementary tables & code.

Lecture 6.2 – Writing an Evidence Table

Introduction & Benefits

  • Converts narrative or quantitative synthesis into an immediately scan-able format.

  • Promotes clarity, organisation, and analytic rigor.

  • Enables rapid identification of patterns, themes, and outliers.

Designing the Table

  • Step 1 – Layout

    • Decide minimal yet sufficient columns (avoid clutter).

  • Step 2 – Software Choice

    • Choose tools offering sorting & conditional colour coding.

    • Automate bibliographic fields via citation-manager exports when possible.

Populating the Table

  • Efficient Extraction Workflow

    1. Skim article → mark candidate data.

    2. Cross-verify with co-reviewer.

    3. Enter into table; use data-validation lists to maintain consistency (e.g., predefined design categories).

  • Consistency & Accuracy

    • Explicitly record scale units (5-pt vs 20-pt).

    • Double-data extraction to reduce transcription errors.

Using Tables for Analysis

  • Identify Patterns → e.g., majority of CBT studies report d\approx-0.60 (moderate effect).

  • Spot Discrepancies → Investigate outlier study with negligible effect; maybe short intervention or different anxiety subtype.

  • Generate Hypotheses → Effect stronger in group formats than individual sessions?

  • Prepare for Meta-Analysis → Table becomes source file for statistical software (RevMan, R \textit{meta} package).

Examples & Best Practices

  • Study published meta-analyses for formatting cues (e.g., Journal of Consulting & Clinical Psychology).

  • Use footnotes to define abbreviations; maintain consistent decimal places for effect sizes.

  • Employ reader-centric design – concise wording, white space, visual anchors.

Common Pitfalls

  • Overly dense tables (information overload).

  • Inconsistent terminology ("pre-test" vs "baseline").

  • Neglecting to update table when re-analysing or excluding a study.

Ethical, Philosophical & Practical Considerations

  • Ethical: Accurate synthesis prevents clinician misguidance; misreporting can harm patients.

  • Philosophical: Reflects epistemological humility – recognising knowledge is provisional and context-bound.

  • Practical: High-quality evidence tables speed up grant writing, guideline development, and replication planning.

Connections to Prior Course Content

  • Builds on Week 4’s systematic search strategies and Week 5’s critical appraisal skills.

  • Draws statistical concepts from earlier modules (confidence intervals, variance, weighting).

  • Reinforces research-integrity principles introduced in Week 1 (transparency, reproducibility, FAIR data).

Conclusion & Next Steps

  • Mastery of narrative, quantitative, and mixed-methods synthesis enables psychologists to craft robust, transparent, and actionable evidence reviews.

  • Evidence tables translate those syntheses into user-friendly artefacts for clinicians, policymakers, and researchers.

  • Practice Task: Create a synthesis & evidence table for 5 recent RCTs on mindfulness-based stress reduction; evaluate heterogeneity and draft a narrative interpretation.

  • Continue refining skills using PRISMA flowcharts, advanced meta-analytic software, and peer de-briefing for bias checking.