How AI and ML Will Save Lives in Healthcare, Episode 2

Ep 2 Clinical Deterioration on General Wards

Most patients who die unexpectedly in a hospital do not die in the ICU. They die on general wards, after hours of warning signs nobody connected. About 292,000 American adults have a cardiac arrest in the hospital each year, and in a registry study of 7,851 of them, 59.4% had at least one abnormal vital sign in the 1 to 4 hours before it happened. The signal is usually there. It is just spread across a dozen numbers charted at different times by different people.

This is the second episode of a series on where AI and machine learning actually save lives in healthcare, one clinical area at a time. This article gives you the ideas and the math. The full deep dive, with the code, the simulations and the study design, is on Substack.

The program that saved 520 lives a year

Kaiser Permanente Northern California built a model called the Advance Alert Monitor. Every hour it estimates each ward patient’s risk of deteriorating, meaning an unplanned ICU transfer or death, over the next 12 hours. Kaiser switched it on across 19 hospitals in a staggered order between 2016 and 2019.

Among patients who reached the alert threshold, 30-day mortality was 15.8% where the program was live and 20.4% where it was not yet live, an adjusted relative risk of 0.84. Kaiser estimated about 520 deaths averted a year.

Two design choices matter more than the model. Kaiser did not show the score to bedside staff; alerts went to a small team of remote nurses who reviewed each chart and called the ward. And the threshold was set so the system produced about one new alert per day for every 35 patients. The model was built around what the team could actually handle.

Your response team is a queue

Every alert lands on a person with a finite day. If alerts arrive at an average of λ a day and the team can review K, the chance the team is overwhelmed on any given day is:

P(overflow) = P(N > K), where N ~ Poisson(λ)

Planning capacity equal to the average load sounds sensible and fails often. A 350-patient ward population at Kaiser’s alert rate produces 10 alerts a day. A team that can review 10 a day is overwhelmed on 41.7% of days. At 12 a day it still overflows 20.8% of the time.

Turn the equation around and it gives you an alert budget. A team that reviews 12 a day and accepts overflow on no more than 5% of days can absorb an average of 7.69 alerts a day. That budget, not the model’s AUROC, is what every threshold has to respect. The full article includes the code for both directions.

Machine learning against NEWS2, at the same workload

The standard to beat is NEWS2, the UK’s National Early Warning Score, which adds up points for abnormal vital signs. In a synthetic cohort of 8,000 ward patients, a gradient boosted model reached an hourly AUROC of 0.826 against 0.792 for NEWS2.

That headline hides where the difference lives. At high alert volumes the two catch almost the same patients. The model’s edge appears only at the strict thresholds a real team can afford. Fit both to a team that reviews 12 alerts a day: NEWS2 can only move in whole points, so its best fitting threshold catches 50.3% of deteriorating patients, while the model, tuned exactly to the budget, catches 56.9%. The next NEWS2 threshold down would overwhelm the team on 74.8% of days.

That is about 7 more patients caught out of every 100, at nearly the same workload. A real gain, and a modest one. The bigger lever was fitting the threshold to the team.

The success paradox

This is the part almost nobody plans for. A deterioration model that works prevents the outcomes it was trained to predict. Rescued patients never deteriorate, so when you retrain on post-deployment data, they look like false alarms. The new model learns that its own warning signs are less dangerous than they are.

A simulation shows the size of it. After one routine retrain, a model that had correctly estimated a 51.4% risk for the sickest 5% of patients now predicts 32.9%. Because most alerts fire on an absolute risk, the same alert cut flags 3.4% of patients instead of 8.0%, and the share of would-be deteriorations it catches falls from 42.5% to 23.3%.

On a dashboard this looks like good news: fewer alerts, calmer wards, precision holding steady. Patients who would have been caught simply are not. The fix is simple once you know to look for it, and the full article walks through it.

What’s in the Full Article

• The capacity equations, with code to size alert budgets for your own census and team
• NEWS2 implemented in code, and a matched-volume comparison against a machine learning model
• A synthetic ward cohort and training pipeline
• The success paradox simulation, and how to retrain without teaching a model its successes were false alarms
• An executive brief, and the economics: deaths averted, ICU admissions avoided and dollars per hospital, with a model you can rerun
• How Kaiser proved its result, and how to copy the design

It is written for clinical data scientists, informatics teams, nursing leaders and health system executives who own a deterioration alert, or are about to buy one.

Read the full deep dive on Substack → HERE

Paid subscribers get the full article and key code. Team Licensing members get the complete code, trained models, implementation plan and reference architecture for running this inside a health system.

About the Author

Brian James Curry is a Kansas City AI researcher, applied scientist and data scientist working in machine learning, causal inference and healthcare intelligence. He is the founder of Vector1 Labs, an independent applied AI research lab, where his Causal Care series applies causal machine learning to health system decisions: which programs work, for whom, and what they are worth. He also founded KC AI Lab, which began in 2016 as the Kansas City Machine Learning Group and has grown to more than 1,500 members.

He has more than twenty years of experience leading analytics, data science and machine learning at Hallmark, Garmin, Koch Industries, Tractor Supply Company and Vail Resorts. His open-source research tools on GitHub include PyCausalSim (causal discovery through simulation), Papilon (marketing mix modeling and complex systems) and tincast (forecasting for AI-critical commodities), along with MeaningFlow and Daedalus. A Kansas State University alum from a small Kansas farming town, he writes about agentic AI, causal inference, healthcare AI and the economics of AI on Medium and Substack. Contact: brian@vector1.ai.