A LHS Approach to CDS Measure Development and Evaluation

A set of Q&A flashcards covering key concepts from the lecture on learning health systems, evidence-based CDSS development, CDS architecture (front-end vs back-end), evaluation, and knowledge checks.

What does LHS stand for in this lecture, and what is its role in CDS measure development and evaluation?

Learning Health System; a framework that embeds new knowledge into daily practice to continually improve care, with CDS as a driver of ongoing improvement.

What is the standard sequence for developing an evidence-based CDSS as described in the talk?

Define problem and desired outcome; search for best evidence; analyze validity and usefulness; integrate with existing systems; evaluate outcomes; retire poor performers; adapt for clinician expertise or patient preference; disseminate results (successes, failures, unintended consequences); ensure appropriateness for the environment.

What is the purpose of standards-based CDS and which resource is mentioned?

Standards facilitate successful integration of clinical guidelines into practice; CDS Connect by AHRQ is referenced as a repository of CDS artifacts.

How is a Clinical Decision Support System (CDSS) defined in the notes?

A computer-based program that analyzes data in EHRs to provide prompts and reminders to help healthcare providers implement evidence-based guidelines at the point of care.

What are some front-end CDS tools and back-end system capabilities?

Front-end: interventions available to end users (alerts, reminders, order sets, relevant displays, workflow aids); Back-end: discrete system capabilities (alert triggers, data input elements, end-user notification methods, integration logic).

In the insulin dosing example, what data and processes are necessary for reliable CDS?

Accurate patient data (weight, labs), up-to-date knowledge libraries (guidelines, normal ranges), templates that trigger appropriate actions (ordering/dispensing/administering) for the right person at the right time, with allowances for clinician modification (e.g., edema, nutritional status, genomics).

What is the CDS Checklist and what are its major items?

A set of checks including Standards, Transparency, Update mechanisms, Workflow integration/Usability, Event-driven vs non-event-driven, and considerations of individuals and populations.

Define sensitivity and specificity in the context of CDS alerts.

Sensitivity (true positive rate): alerts fire when the condition is present; Specificity (true negative rate): alerts do not fire when the condition is absent.

What is shadow mode testing in CDS deployment?

Running CDS in a live environment without displaying alerts to users to analyze when and where alerts would fire, informing sensitivity and specificity without impacting care.

What are common unintended consequences illustrated by allergy alerts?

Alert fatigue, excessive overrides, incorrect or excessive allergy data, wasted clinician time, suboptimal drug choices, and overall reduced usefulness of CDS.

What does CDSS Evaluation typically assess?

Back-end and front-end performance, patient outcomes (mortality, morbidity, QoL, LOS), improvements in healthcare processes, disparities, user workload and efficiency, implementation costs, provider use, and user satisfaction; distinguishes Working as designed vs Working as intended.

What does Knowledge Check 1 emphasize about adopting CDS?

Review current literature and user groups to find if a CDS with successful outcomes already exists before building a new one.

What does Knowledge Check 2 emphasize about evaluating a new antibiotic ordering alert?

Focus on the percentage of alert overrides and override reasons to gauge alert volume and potential fatigue, rather than only overall outcome changes.

What are the key points about developing and evaluating CDS measures according to the lecture?

Build on existing knowledge and contribute new knowledge where gaps exist; evaluate whether measures achieve outcomes without undue burden; test in shadow mode; ensure information is delivered to the right user at the right time.

What is AHRQ CDS Connect?

A platform by the Agency for Healthcare Research and Quality that provides CDS authoring tools and a repository of CDS artifacts—actionable medical knowledge translated into computable, interoperable decision support.