Latest News

High-Value Opportunities for Multimodal AI in Clinical Care and Research

By Deborah Borfitz 

February 4, 2025 | Amid an industrywide shortage of workers and hospital capacity, a multimodal artificial intelligence (AI) platform providing real-time visibility into the clinical picture for patients is finding a national audience with health systems big and small who want to “make the right thing the easy thing to do” for their overburdened clinicians. The award-winning technology integrates seamlessly with all major electronic medical record (EMR) systems to help predict costly complications such as sepsis, deterioration, and pressure ulcers so frontline providers can prioritize attention to high-risk cases, says Suchi Saria, Ph.D., professor of engineering and public health as well as director the AI and Healthcare Lab at Johns Hopkins University. 

Saria is also founder and chief executive officer for Hopkins’ spinout Bayesian Health, developer of the AI platform that TIME named among its “Best Inventions of 2023 and 2024.” This came on the heels of a series of peer-reviewed articles published a year earlier, highlighting the company’s Targeted Real-Time Early Warning System (TREWS) and its groundbreaking results for sepsis. 

The peer-reviewed articles were based on a first-of-its-kind study demonstrating real-world use and impact of the platform, which made the cover of Nature Medicine in 2022. Participants in the main study (DOI: 10.1038/s41591-022-01894-0) included close to 7,000 patients with sepsis among nearly 600,000 individuals being monitored by TREWS across five Johns Hopkins hospitals. 

Sepsis is the leading cause of hospital death and earlier sepsis intervention is associated with improved patient outcomes and decreased mortality. “Every hour of delayed treatment matters,” Saria stresses. 

Among the nearly 10K sepsis cases analyzed, the system correctly identified 82% of positive cases (DOI: 10.1038/s41591-022-01895-z), which is a significant improvement relative to the typical early-alert system, which has a sensitivity of between 20% and 30%. Moreover, patients were identified 5.7 hours prior to previous sepsis identification and treatment while demonstrating a sixfold reduction in the false alarm rate, which has traditionally been a barrier to the adoption of EMR-based alerting tools. Remarkably, physician adoption of TREWS was sustained at over 90% for more than two years and deaths from sepsis fell by nearly 20%. 

And this was at Johns Hopkins, widely regarded to have one of the best hospitals in the world. At subsequent implementations elsewhere, says Saria, “we’re seeing similar or better results.” 

‘Learning Moments’ 

The Bayesian Health platform improves patient outcomes through “a dramatic ability to streamline workflows” which significantly reduces the burden on clinicians. Optimizing workflow was most notable for nurses who were able to “do more with less,” Saria says. Nurses have historically been asked to perform multiple clinical assessments (e.g., mobility, pressure ulcer, and deterioration) which ask many of the same core questions, several times a day throughout a patient’s stay, resulting in a pileup of repetitive work and replicate documentation. 

“That’s like giving nurses 30 hours of work in a 12-hour shift,” says Saria. AI helps “smartly identify” which patients are susceptible to life-threatening complications like sepsis and need their undivided attention, based on what the algorithm has learned from ground truth—retrospective patient record reviews. This creates real-time “learning moments” for clinicians that literally save lives.  

Once episodes of care are complete, AI quickly sums up what happened and what was and wasn’t done, Saria says. That data is then used by quality improvement teams to understand “where the gaps and barriers to better care are” without having to do manual chart reviews.  

It’s all about making the right decisions for patients, she says. As this kind of infrastructure scales nationally over the next few years, “there is an opportunity to build the infrastructure for life science companies and health systems to partner” so that clinical trials get populated with the right number and type of patients for a particular protocol. 

Currently, recruitment of patients to clinical trials depends on clinicians remembering the criteria for patients to qualify for a clinical trial and “manually” connecting the dots of identifying eligible patients and recruiting them to participate. Because Bayesian Health’s platform sits within the workflow of physicians and has high adoption, it could help “close the loop” between companies struggling with recruitment into their studies and health systems treating would-be participants who may benefit from their interventions, says Saria. Both providers taking care of patients and those helping recruit patients for trials would be empowered to “take the right actions in real time.” 

Saria adds that the worlds of drug development and care delivery need to “combine and converge” for the sake of clinicians and patients. Embedding advanced AI algorithms within EMRs is an easy and efficient way to “make those two worlds meet.” 

Saving Lives 

Saria was prompted to turn her years of sepsis research into a clinically impactful sepsis screening tool after losing her 26-year-old nephew to this serious complication, which occurs when the body responds improperly to an infection. The broader mission of Bayesian Health is to enable care that is proactive and safer while reducing clinician burnout, she says.   

The long-term stress, extensive hours, and lack of work-life balance facing providers today has taken the joy out of practicing medicine. Many people who went into the profession because they wanted to take care of patients are now leaving the workforce feeling overwhelmed and exhausted.  

Bayesian Health’s technology uses AI to read multi-modal clinical data in the background, summarize it, and put actionable information at the fingertips of clinicians in real-time, so they can intervene to positively impact the trajectory of patients, continues Saria. Predictions made by the system come with a clear explanation, and clinicians are encouraged to add their own impressions and interact in a collegial, consultative fashion. The technology’s real power is in facilitating human-machine teaming to achieve the best care at the best time for patients. 

The targets, in addition to sepsis, are pressure ulcers and all-cause deterioration—for example, unplanned but preventable transfers to the intensive care unit that are associated with increased hospital mortality, morbidity, and length of stay, she says.    

Many opportunities exist for better anticipating patient needs during transitions of care, especially when patients leave the hospital, to mitigate the risk of adverse events post-discharge and thereby reduce readmissions, Saria says. Bayesian Health also has a collection of other high-value modules on the roadmap for the platform that can help improve patient outcomes and experience, reduce lengths of stay and complications leading to mortality. 

Unlike how most startups launch, Bayesian Health has come out of the gate in a way more typical of a new drug or medical device—first, demonstrating clinical utility by running clinical trials. Its platform was launched at the five Baltimore-area hospitals in the Johns Hopkins network to demonstrate how the technology offers a meaningful improvement to the early and accurate detection of sepsis, relative to “best practice” alerts that currently pop up within the EMR, says Saria.    

Bayesian Health has since partnered with numerous other health systems, as will soon be publicly announced, she adds. The benefits being seen in smaller community systems have been particularly impressive with reduction in sepsis-related length of stay by several days versus a half day in the original trials at Hopkins. With later deployments of Bayesian’s sepsis product, the sensitivity has also inched up closer to the 90% mark. 

Response time to the alerts has also been dropping, says Saria. Initially, it would take clinicians as long as two hours to act. This is now down to under one hour in some locations with the expectation of getting closer to the 30-minute mark soon. When even an hour makes a difference, this means actionable insights for providers and meaningfully better outcomes for patients.  

Reducing Pressure Ulcers 

In a subsequent trial at Johns Hopkin, TREWS was used to target pressure ulcers—a painful ordeal for patients and the second leading cause of malpractice risk for hospitals, notes Saria. TREWS demonstrated a marked improvement in the early and accurate detection of this potentially deadly skin injury, which was driven, much like the sepsis use case, by a dramatically streamlined workflow. 

The system helped nurses identify the one out of eight patients truly at risk for developing a pressure ulcer, she reports. It also identified 95% of the pressure ulcers present on admission. The comparison point here is not any sort of EMR-embedded alert system, but the cumbersome practice of asking nurses to perform a visual pressure ulcer risk assessment twice a day on every hospitalized patient.  

Moving to the Bayesian Health screening tool could effectively give nurses “a tenth of the work” by having them actively monitor at-risk patients and intervening prophylactically to prevent them from developing pressure ulcers. “Early intervention leads to better outcomes... and less medical waste,” she stresses. 

Patients who acquire a pressure ulcer end up staying in the hospital much longer than they otherwise would at a huge cost to the overall healthcare system. They also add to the time burden on the nurses caring for them. 

Improving Quality 

In addition to solving the right problems in healthcare, AI needs to be “above the bar,” delivered in a way that’s easy to use, and have an appropriate level of governance from both a technical and policy perspective, says Saria. On the first point, she cites the clinical labor shortage that a McKinsey report says could create $170 billion in incremental costs by 2027. 

This is on top of other confounding related issues, including studies finding one in 10 patients are harmed by medical mistakes, adds Saria. Diagnostic errors are one of the leading causes of death in the U.S., and real-time tools powered by AI could potentially help prevent them. By reducing the risk of misinterpretation or errors in the EMR, ambient documentation has gained a lot of traction in healthcare, she notes. 

Transformer-based AI devices take this one step further, listening in on conversations between physicians and their patients and automatically transcribing that speech to create a transcript, she continues. Another AI bot then converts that into a summary document. 

According to a recent survey by the Medical Group Management Association, 64% of doctors now use visit transcription or speech recognition, Saria reports. This has effectively removed some of the “fear factor” surrounding AI in the healthcare space. 

EMR provider Epic, implemented in many health systems nationally, also has an In-Basket secure messaging system clinicians are using to interact with patients, she says. Here, AI is being used to automatically know text is coming in and create an automated response. 

Some health systems are starting to have physicians evaluate if those responses are “good enough” and, if not, make edits before sending them out, Saria says. Patient-facing live correspondence applications come with slightly increased malpractice risk if incorrect and potentially harmful information is inadvertently transmitted. 

Bayesian Health is part of a next-generation, transformer-based movement where AI is ingesting multimodal datasets to do clinical reasoning, says Saria. This contrasts with ChatGPT and the messaging system of an EMR, which deals only with text data. 

What AI is doing with the data is far more advanced and high stakes than “superficial pattern matching” required for low-risk administrative tasks, like simple ambient dictation or cut-and-paste Q&As, Saria adds. Outside of clinical medicine, a great deal of work is underway to use multimodal data—to, for example, generate a video from a transcript or, as OpenAI and Google are doing, creating AI assistants that can converse with users more naturally and intuitively. 

Massive multimodal data is needed to properly power the models in healthcare where the inputs can include heart rate, blood pressure, respiratory rate, images, and unstructured medical notes, says Saria. Much progress has been made on the technical front, she adds, referencing a mixture-of-experts deep learning, transformer-based architecture known as FuseMoE that was designed for multimodal fusion.   

“We’re constraining these advanced models to focus on improving quality,” she says, rather than using generative AI to tell a story. The focus is on predictive tasks where the ask is “more yes/no or a grading question like high, medium, or low risk” and data-based facts. 

Human-Machine Teaming 

Governance of AI is where some of the bigger open gaps remain, says Saria. The perception is that clinical applications are high risk, but “there is almost no clarity around evaluating and overseeing” them. 

Bayesian Health is going through the regulatory process with the U.S. Food and Drug Administration (FDA) to help fill those gaps and establish trust and oversight, she says. Congress passed a regulation in 2023 allowing the FDA to implement Predetermined Change Control Plans, enabling approved AI medical devices to “continually update and learn over time... as it sees more data.” Additionally, much work has been happening around the building of human-in-the-loop monitoring infrastructure to address the inevitable “drifts and shifts” of data and the application of AI over time when applied in the real world, says Saria. Some have proposed the building and testing of these kinds of monitoring systems so they can be deployed nationally. 

As Saria and her colleagues discovered during 2022 interviews with clinicians, the key to AI adoption is human-machine teaming (npj Digital Medicine, DOI: 10.1038/s41746-022-00597-7). Clinicians using the Bayesian Health platform reported feeling they were “partnering with the technology” rather than having their clinical judgement replaced.

 

Load more comments
comment-avatar