Hospital data models

Multimodal AI in biomedicine - lecture 4

Static prediciton models, longitudinal models, mutlimodal models, patient-specfic models and digitial twins

• Static prediction models, which make predictions from a single snapshot of patient data

• Longitudinal models, which learn from patient trajectories over time

• Multimodal models, which combine structured and unstructured patient information

• Patient-specific models and digital twins, which seek to represent the evolving state of an individual patient

TREWScore paper

• predict future septic shoch

• EHR data into combined score from 27 factors

• each patient is a fixed vector of 27 vitals and features → snapshot of current state, not a learned sequence embedding

• current phyisiolgy and recent changes, loses notes, treatments and interventions

• supervised retrospective label learning

• AUC and sensitvity for evaluation

• based on a signle hospital and an old cohort

• weak generalization

Multimodal ICU mortality model paper

• predict in-hospital death after first 24 hour stay

• static variables, hourly time series, clinical notes and chest X-rays

• representations: longitudinal and multimodal, snapshot at hour 24: one embedding per modality, then fusion

• captures everything in the first 24 hours and loses everything after 24 hours

• one encoder per data type, embeddings are later fused

• AUROC and AUPRC evaluation

• uncertainty, bias and failures: fairness, age, notes, data splits

• strongest generalization

clinicalT5 readmission model paper

• predict all-cause 30-day readmission

• hospital module and diagnosis text

• one vector per admission, notes are split into 200 words chunks and embedded; static, not timeline stays and no history for admission

• text content and basic facts stay, loses: history, medicatoins, labs

• encoder-decoder LLM produces text embeddings

• accuracy, predicion, recall, specificity, etc. for evaluation

• no fairness analysis, no comparison with a structured baseline

• not really generalization

What info about a patient would you want a prediction model to know before making a clinical prediction?

demographis, gender, medical history, age, notes, heart rate, vital signs, drug use, lab measurements, etc.

One chain for upstream models order?

1. clinical data

2. patient representation

3. model architecture

4. prediction

5. evaluation

6. robustness - uncertainty - generalization

Two model see the same lab test. Model A uses only the latest value of each test; Model B uses all values over time. What can Model B use that Model A cannot?

ho fast and in which direction each value is changingw

Heart rate is recorded every few minutes, a blood gas only when a doctor orders one. If we fill the gaps by carrying the last value forward, what is the biggest risk?

A 10-hour-old value looks current, and the fact that no new test was ordered is lost

What additional information might clinical notes provide that is difficult to capture in structured EHR variables?

visual observations, history, pain

Combining more modalities will always improve predictive performance

False

A model has an AUC of 0.9 on an internal test set. What does this tell us?

It discriminates well on that evaluation population

Two model have an identical AUC. What additional information could help determine which one gives more trustworthy risk estimates?

calibration

internal, temporal, external generalization

internal - same hospital, same period

temporal - same hospital, later period

external - other hospitals, countries

A model performs very well in hospital A but worse in hospital B. Which explanation should we consider?

• dataset shift

• different patient population

• different clincial workflows or measurement practices