Comprehensive Notes on Ultrasound Research (POCUS)

  • Introduction

    • Point-of-care ultrasound (POCUS) has rapidly evolved from a novel technology to an essential component of medical imaging and clinical care across specialties.
    • Drivers of success: technological advancements, evidence-based medicine emphasis, standardized training programs, and certification processes.
    • Clinician scientists and researchers have driven progress by testing, validating, and assessing POCUS applications on patient care.
    • Growth factors: more academic positions, training modules, and fellowships in emergency medicine and beyond.
    • With increasing advanced emergency medicine ultrasound fellowships and research requirements for accreditation, there will be greater urgency for new research ideas and proposals to IRBs and publication.
    • High-quality research > quantity: disseminating high-quality, clinically relevant findings can impact patient-centered outcomes and practice across disciplines.
    • Fellows and faculty should understand unique challenges in POCUS research and the dynamic between ultrasound technology and expanding applications.
    • Research skill workshops can help build study design, methods, data analysis, and networking.
    • Collaborations broaden publication portfolios; small projects can still enrich a portfolio.
    • Early career practice: regularly reassess research portfolio, progress, and achievements at least every 3 months to gauge impact and adjust strategies.
    • This chapter guides fellows, faculty, and researchers in navigating a successful path in POCUS research; it emphasizes essential principles, practical tips, and focused strategies; it does not cover epidemiology, biostatistics, research ethics, or regulation in depth (those topics are covered elsewhere).
    • Scope: main principles to cutting-edge advancements; includes examples from collective experience, successes, and mistakes.

  • The Landscape of POCUS Research

    • POCUS and Patient-Centered Outcome Research

    • POCUS is often seen as diagnostic; researchers should also understand its potential positive impact on overall patient outcomes.

    • Traditional patient-centered outcomes (mortality, return of spontaneous circulation, neurological status) are common but may not capture the specific impact of POCUS.

    • There is a scarcity of patient-centered outcome research in POCUS because many studies treat POCUS solely as a diagnostic tool rather than an intervention.

    • Some outcome studies show no significant impact, often due to choosing suboptimal outcomes.

    • Consider a broader range of outcomes: procedural safety, reduction in morbidity, time to treatment, downstream imaging, throughput improvement, and patient satisfaction.

    • Ultrasound-guidance for procedures (e.g., vascular access) is a strong example of a POCUS application positively affecting patient-centered outcomes and is featured in evidence-based guidelines.

    • Despite benefits in CPR and polytrauma, POCUS has not yet been fully integrated into mainstream protocols like ACLS/ATLS; more evidence on efficacy and cost-effectiveness is needed to drive integration.

    • There is a need for high-quality, multi-institutional, large-scale prospective POCUS research to address these challenges.

    • When developing projects, prioritize patient-centered outcomes to improve funding potential and impact.

    • Comparative Diagnostic Accuracy in POCUS Research

    • Compared with plain film radiography and CT, POCUS offers real-time bedside diagnosis, potential downstream testing reductions, and fewer invasive procedures.

    • Observational studies largely address diagnostic accuracy; their impact on patient-centered outcomes remains uncertain.

    • Focusing solely on diagnostic accuracy or feasibility may not yield broad impact; broader questions about patient management and healthcare delivery are needed.

    • Diagnostic accuracy remains meaningful in critical illnesses where other imaging is not feasible (e.g., trauma, cardiac arrest, septic shock, undifferentiated hypotension) where there may be no alternatives.

    • After establishing accuracy across multiple operators, shift toward broader implications: patient management, treatment decisions, procedural outcomes, and healthcare delivery.

  • POCUS Research in Healthcare Delivery

    • Cost-effectiveness and workflow: reporting on cost-effectiveness of specific POCUS applications and their impact on clinical workflow informs adoption and integration.
    • Research on procedural safety and throughput provides insights into optimizing patient care and resource use.
    • Focus areas: integration into workflows to improve disposition, shorten length of stay, reduce imaging/ancillary testing, cut costs, and optimize resource use.
    • Expanding POCUS focus to facilitate service delivery in settings with limited access to 24-hour radiology; outpatient POCUS could reduce referrals and expedite care, with attention to patient- and system-level metrics especially in value-based care environments.
    • Integration into healthcare systems in lower-resource and austere environments can broaden access to quality care.
    • Implementation science is important to promote knowledge translation and evidence-based practice in POCUS.

  • POCUS Educational Research

    • Studies have explored education and training programs to improve image acquisition, interpretation, and clinical decision-making.
    • Training platforms include: structured learning modules, hands-on sessions, preceptorships, gamification, simulation, and virtual reality for physicians-in-training, medical students, nurses, and non-medical providers.
    • When evaluating educational interventions, consider:
    • Educational framework as the theoretical underpinning
    • Uncontrolled environments with potential confounders (e.g., concurrent training, varying learning opportunities)
    • A broad range of outcomes (learner preferences, knowledge, skills, attitudes, behaviors, healthcare results)
    • Mixed or diverse research methodologies (qualitative, quantitative, mixed-methods)
    • Well-developed surveys/assessments with validity evidence
    • Rater training and assessment of inter-/intra-rater reliability
    • Longer-term outcomes (retention, practice application) beyond initial knowledge/skill gains
    • Given these complexities, engaging an education research expert early is advisable to improve rigor and avoid common errors.

  • POCUS Technology Research

    • As POCUS tech evolves (handheld devices, tele-ultrasound, AI integration), methodologies must adapt.
    • Handheld devices enable imaging in EDs and remote settings; research includes diagnostic accuracy and clinical outcomes with handhelds and evaluating tele-ultrasound.
    • AI in POCUS can address image interpretation, automated measurements, predictive models, and real-time decision support; multi-institutional, multidisciplinary collaborations can enhance statistical support, equipment access, and multicenter studies.
    • Large databases enabling data centralization across institutions support large-scale studies with diverse populations and standardized protocols.
    • Practical considerations: stay current with technology trends, understand capabilities/limitations of new devices, and explore AI's role in improving diagnostic accuracy.
    • Technology-driven research should prioritize patient-centered outcomes and real clinical impact; industry sponsorship is possible but may require balancing with clinical outcomes; be mindful that purely technological improvements without demonstrated patient outcomes may be less fundable.
    • Collaboration with industry can provide access to technologies and future developments; success depends on combining clinical insight with technical expertise.

  • Choosing a Research Topic

    • The backbone of any project is a clear research question; develop a precise question and tailor it to available institutional resources and expertise.

    • Before finalizing, consult mentors/advisors across and beyond ultrasound to optimize clarity, focus, significance, and broader impact; these discussions can improve publication potential.

    • POCUS research has unique methodological challenges due to variability in ultrasound protocols, operator skills, machines, and patient populations.

    • Test characteristics can vary with disease progression; enrollment criteria should reflect these limitations.

    • Practical challenges: limited funding, institutional restrictions, the acute care environment's unpredictability, and reliance on ancillary staff; research strategy should be adaptable to diverse settings.

    • The PICO Framework

    • PICO = patient, intervention, comparison, outcome; a structure for framing focused questions and aligning interventions and outcomes.

    • Example: P: Among patients in cardiac arrest, I: does POCUS, C: compared with manual pulse palpation, O: reduce time to resumption of chest compressions?

    • Modifications include PICOT (adds time) and PICOS (adds study design).

    • Why use PICO in POCUS: informs clinical guidelines, trial design (population narrowing, appropriate comparator), and ensuring outcomes are discrete and measurable.

  • The FINER Criteria

    • FINER stands for Feasibility, Interest, Novelty, Ethics, Relevance; a framework to evaluate potential POCUS research topics.

    • Table 25.1 (criteria and tips) highlights:

    • Feasibility: study population, funding, resources, time, institutional support; tips include pilots, small but feasible scope, collaboration, and student assistance.

    • Interest: align with program theme and personal curiosity; broad interest across specialties;

    • Novelty: unique or extension of prior work; potential for publication and future research; explore new methodologies.

    • Ethics: participant respect, harm minimization, consent, data privacy, equity of care; ensure ethical integrity.

    • Relevance: alignment with POCUS scope and potential to affect outcomes and resource use; ensure patient-oriented evidence and alignment with priorities.

    • Table 25.2 provides examples of how potential projects might align with FINER criteria (illustrative ratings across criteria).

    • Before selecting a project, ensure it matches educational objectives and program strengths; choose teammates with aligned interests; aim for topics with clear relevance and consensus; consider generalizability and patient-centered impact.

  • Before Starting a Research Project

    • Planning requires patience, careful planning, attention to detail, resilience, and transparency.
    • Writing can sharpen critical thinking; the background, methods, and discussion develop as you review literature and refine the question.
    • Literature searches guide idea generation and help identify limitations/controversies and under-researched areas to refine the question.
    • Seek feedback from mentors and peers to refine focus and broad significance; ensure IRB requirements are considered early for feasibility and ethics.
    • Recognize unique POCUS challenges: variability in scan protocols, operator skill, machine differences, and diverse patient populations; plan enrollment accordingly.

  • Team-Building, Authorship, and Research Mentorship

    • Team-Building

    • Build a unified team with shared research/publishing goals; cohesive teams boost productivity.

    • Structure: define roles, assign task ownership (proposal writing, enrollment, database design, statistics, manuscript writing); include stakeholders impacted by the study (e.g., residents as co-investigators).

    • Start with smaller teams to avoid scheduling delays; teams expand as the project progresses.

    • Include statistical support; consult institutional biostatisticians when needed.

    • A team charter plus an updated task list and timeline help maintain progress and accountability.

    • A practical approach: schedule regular meetings with milestones and action items; plan for sponsor-driven timelines.

    • Authorship

    • Communicate authorship expectations at planning; revisit before submission.

    • Identify first, second, and last authors; other authors listed by role or alphabetically.

    • The last author is typically a senior faculty member overseeing the project.

    • Consider recognizing contributors who underrepresented groups or early in their career; adhere to ICMJE criteria for authorship:

      • Substantial contributions to conception/design or data acquisition/analysis/interpretation;
      • Drafting or critically revising the manuscript;
      • Final approval of the published version;
      • Accountability for all aspects of the work.

    • Acknowledgements for those not meeting all four criteria.

    • Reassess authorship as the project evolves; avoid gift or ghost authorship.

    • Research Mentorship

    • Mentors guide novices through research steps; a mentee can have multiple mentors for different aspects.

    • Establish a contract-based relationship with clear roles, commitments, goals, timelines, and expectations.

    • Regular meetings, timely feedback, and mutual accountability are essential; SMART goals recommended.

    • Ideally, mentees find mentors with personal/professional investment in their development.

    • Seek mentors beyond one institution or specialty; digital platforms enable external mentoring; peer mentorship is valuable for statistics, informatics, and study design expertise.

  • Research Process Steps

    • Developing a Project Timeline

    • Each project is unique; activities may overlap; a timeline helps organize tasks and accountability.

    • Table 25.3 provides an example POCUS research timetable with shaded periods representing phases; timelines can be condensed or expanded based on scope and deadlines (e.g., grant performance windows, abstract deadlines).

    • Plan for regular updates and milestones; monitor progress and adjust as needed.

    • Writing a Study Protocol

    • Start with a literature review to identify similar work and barriers; build on prior work to address gaps.

    • Align protocol with reporting guidelines (Table 25.4).

    • Core elements (Table 25.5): objectives, methodology, expected outcomes, data collection, analytic plan; include a clear hypothesis and literature-based rationale.

    • Define the POCUS application, patient population, inclusion/exclusion criteria, image acquisition protocol, equipment/probe settings, and user groups.

    • Explicitly define POCUS training for study staff to ensure consistency.

    • For multi-institution studies, specify data collection, transfer, and storage methods; include data-sharing agreements when needed.

    • Provide overall sample size with justification (power 80% or 90%, alpha = 0.05, anticipated incidence or effect size); include a 10–20% buffer to account for dropout and inconclusive results.

    • A well-crafted protocol guides the team, IRB approvals, and funding.

    • Consider a very brief pilot study on at least one sample subject to test feasibility; data from pilot subjects should not be included in final analyses.

    • IRB (Institutional Review Board)

    • Before submission, prepare IRB documents with required protocols, consent forms, and information; align with local IRB criteria; consult ultrasound division/research directors for alignment.

    • IRB review ensures ethical standards, patient safety, and informed consent; address non-invasiveness of POCUS, incidental findings, and minimal risk.

    • Write the IRB proposal in clear, simple language to accommodate varied reviewers.

    • Typical IRB review types and timelines (Table 25.6):

      • Exempt: minimal risk; 2–4 weeks
      • Expedited: minimal risk; 1–2 months
      • Full board: more than minimal risk or interventional studies; 2–3 months

    • Use IRB downtime to begin manuscript writing; prepare for revisions after review.

    • Data Collection

    • Data collection should be systematic from image acquisition to storage/transfer (e.g., DICOM, MP3, AVI).

    • Train the team in POCUS techniques to capture high-quality images; specify who obtains images and their training; if secondary image review is involved, describe the rubric and reviewers.

    • Use an online data collection database to facilitate data extraction and sharing with statisticians; ensure data collection staff are trained and consistent.

    • POCUS data collection is unique due to reliance on actual ultrasound images; standardize ultrasound protocols and proficiency; plan for data storage and transfer.

    • Maintain accurate records of patient demographics, clinical setting, and protocol deviations; monitor data collection for issues (redundant variables, unclear responses).

    • Data should be entered into a dedicated data collection form; consider direct digital entry to streamline analysis.

    • Collaboration with statisticians early helps ensure data are compatible with planned analyses; discuss predictors, confounders, and outcomes early.

    • Data Analysis

    • Involve statisticians early to plan analyses and ensure data are suitable for statistical software; discuss the analytic plan upfront and budget for statisticians’ time.

    • Tables and figures: Figure 25.1 provides statistical analysis pathways; Figure 25.2 illustrates TP, FP, TN, FN; Table 25.7 summarizes common statistical methods; Table 25.8 lists formulas for test characteristics.

    • Table 25.7 (highlights):

      • Descriptive statistics: frequencies/proportions; mean (SD) or median (IQR); decide based on data distribution.
      • Bivariate analysis: categorical comparisons with chi-square; Fisher's exact test if expected counts are <5; continuous with t-tests or nonparametric equivalents.
      • Correlation/association: Pearson for normal data; Spearman for non-normal data.
      • ANOVA for comparing means across >2 groups; check homogeneity of variances and normality.
      • Multivariable analysis/regression: assess associations and predictions; include discrimination and calibration metrics; validate in external data when possible.
      • Survival analysis: Kaplan-Meier and Cox proportional hazards models; check proportional hazards assumption.
      • ROC analysis: evaluate diagnostic accuracy; ROC curve plots sensitivity vs 1-specificity; AUC in [0,1], higher is better; ideal AUC = 1.0.
      • Kappa and agreement: interrater reliability; use intraclass correlation coefficient (ICC) with more than two raters; consider alternatives like Gwet's AC1 when kappa is biased.

    • Understanding imaging data: clinicians must communicate potential variability in image interpretation to statisticians for appropriate methodologies.

    • Emphasize that statistical analysis of image-based data has evolved with computational statistics, machine learning, and software tools; anticipate more user-friendly tools and algorithms.

    • Clarify statistician’s role upfront (planning, analysis, or both) and document the analytic plan; credit statisticians as authors if their contributions meet ICMJE criteria.

    • Submitting the Manuscript

    • Location planning: search journals focusing on POCUS research (e.g., PubMed/Journal Selector tools) to identify suitable venues.

    • Strategy: identify top five target journals; prepare for resubmission to backup options if rejected; consider alignment with journal scope, decision timelines, impact factor, and publication fees.

    • Beware of predatory journals; assess review rigor, editorial oversight, and credibility before submitting; online platforms can offer rapid dissemination but may lack rigorous review.

    • When in doubt, consult mentors and local researchers before submitting to a journal or alternative platform.

  • Grants and Funding

    • Grants are essential for high-quality, multicenter research and broad impact on patient care; they enable new protocols and technologies and support diverse populations.
    • Focus areas for funding: patient-centered outcomes, cost-effectiveness, and development of novel technologies; projects focused solely on accuracy, educational proficiency, or purely comparative research may be less likely to receive funding.
    • Grant proposal elements are described in Table 25.11:
    • Project summary: concise synopsis of aims, methods, and impact
    • Specific aims: 2–4 aims with hypotheses and expected outcomes
    • Research plan: background, significance, innovation, approach, sample size, data analysis, potential problems/solutions, bibliography
    • Facilities and resources: available support
    • Equipment: essential devices
    • Biosketch: PI and key personnel bios
    • Budget and justification: detailed line-item costs
    • Supplemental forms: data sharing plans, population considerations, study timeline, human subjects protection
    • Letters of support: endorsements from collaborators/institution
    • Funding sources (Table 25.12):
    • Federal:
      • NIH, NIBIB, AHRQ, BARDA, DARPA, DOD, etc.
    • Foundations:
      • AHA, RWJ, SAEM, AIUM, EMF, etc.
    • The Specific Aims Page (Table 25.13) example for POCUS in pulmonary embolism
    • Structure: three parts (topic/need, proposed solution, aims)
    • Aim examples (illustrative):
      • Aim 1: Determine incidence of a composite primary outcome within 5 days in acute PE; risk-stratify by SPESI ESC; compare outcomes across risk groups and report sensitivity, specificity, PPV, NPV, and misclassification proportion
      • Aim 2: Evaluate sensitivity, specificity, PPV, NPV of each RV assessment modality (CT, cardiac POCUS, natriuretic peptide, troponin) for predicting deterioration within 5 days; define abnormal RV dilation for each modality
      • Aim 3: Develop a prediction model for PE patients for the primary outcome of deterioration within 5 days; use logistic regression to identify variables and build an RV-dysfunction-inclusive model; perform internal and external validation
    • Abbreviations: CT = computed tomography; ESC = European Society of Cardiology; PE = pulmonary embolism; RV = right ventricle; sPESI = simplified PE severity index; RVD = right ventricular dilation
    • Non-funded work can build a foundation for funding by generating preliminary data and refining methods; securing federal funding is a long-term commitment requiring persistence and a strong collaborative team.

  • Tables, Figures, and Resources (mentioned in transcript)

    • Table 25.1: FINER criteria and tips for POCUS topic evaluation
    • Table 25.2: Examples of comparing potential research projects against FINER criteria
    • Table 25.3: Example POCUS research timetable (timeline by months/years)
    • Table 25.4: Reporting guidelines by study type and methods
    • Table 25.5: Study protocol summary components
    • Table 25.6: IRB approval types and estimated timelines
    • Table 25.7: Common statistical methods in POCUS studies (descriptions and examples)
    • Table 25.8: Formulas for common test characteristics (Sensitivity, Specificity, PPV, NPV, Likelihood Ratios)
    • Table 25.9: Specific challenges in POCUS research methodology
    • Table 25.10: Journal selector sites (Elsevier Journal Finder, JANE, JournalGuide, etc.)
    • Table 25.11: Grant proposal sections (summary to supplemental forms)
    • Table 25.12: Sources of grants and sponsorship (Federal and Foundations)
    • Table 25.13: Example specific aims page for POCUS in pulmonary embolism
    • Table 25.4 and Table 25.6 provide guidance on reporting and IRB processes
    • Figures 25.1 and 25.2: Statistical analysis pathways and TP/TN/FP/FN assessment

  • Important Formulas and Equations (examples provided in the text)

    • Sensitivity: ext{Sensitivity} = rac{TP}{TP + FN}
    • Specificity: ext{Specificity} = rac{TN}{TN + FP}
    • Positive Predictive Value: ext{PPV} = rac{TP}{TP + FP}
    • Negative Predictive Value: ext{NPV} = rac{TN}{TN + FN}
    • Positive Likelihood Ratio: ext{LR}^+ = rac{ ext{Sensitivity}}{1 - ext{Specificity}}
    • Negative Likelihood Ratio: ext{LR}^- = rac{1 - ext{Sensitivity}}{ ext{Specificity}}
    • ROC/AUC concept: ROC curve plots sensitivity against 1-specificity; AUC ranges from 0 to 1, with 1 being perfect accuracy

  • Practical takeaways for exam preparation

    • POCUS research emphasizes patient-centered outcomes and healthcare delivery implications, not just diagnostic accuracy.
    • A robust POCUS study plan includes a well-defined PICO question, FINER-based topic evaluation, a rigorous study protocol with sample size justification, IRB planning, standardized data collection, and a detailed statistical plan with early statistician involvement.
    • When planning grants, develop a concise Specific Aims page and align with funder priorities; plan for cross-institution collaboration and robust project management; be mindful of predatory journals and the value of traditional PubMed-indexed venues.
    • The FINER framework and careful team-building (with mentors and clear authorship criteria) are critical for successful publication and career development.

  • Ethical, philosophical, and practical implications

    • Balancing innovation with patient safety and equity in access to POCUS resources.
    • Ensuring transparent authorship and proper attribution (ICMJE criteria).
    • Recognizing and mitigating biases (operator dependence, selection bias, variability in protocols).
    • Emphasizing implementation science to translate evidence into practice and improve real-world outcomes.

  • Connections to foundational principles and real-world relevance

    • POCUS research integrates clinical medicine, imaging science, statistics, and health services research.
    • Emphasizes implementation into workflows, cost-effectiveness, and education to maximize patient benefit in diverse settings.
    • Highlights ongoing evolution with handheld devices, tele-ultrasound, and AI, shaping future research agendas and training needs.