Why Most Readmission Models Never Reduce Readmissions
A readmission prediction model can validate beautifully and still prevent almost nothing. One equation shows where the readmissions you hoped to avoid actually leak out, why better AUC barely moves the result, and what to rank patients by instead. Python included.
Brian Curry | Vector1 Research, Kansas City
Most health systems now run a readmission prediction model. It scores every inpatient before discharge, the high scorers get flagged, and a care transitions team calls, visits or schedules follow-up for as many of them as it can. The model was validated, its AUC was reported at the committee meeting, and two years later the 30-day readmission rate looks about the same.
That outcome is the normal one, and it is not a failure of the model. A readmission prediction model answers a narrow question: who is likely to come back? A readmissions program needs an answer to a different question: whose readmission can we prevent, with the staff we have, in the 30 days we have? The gap between those two questions is where nearly all the expected benefit disappears.
This piece is written for population health leaders, the people who own the transitions budget and answer for the readmission rate. It sets up Thursday’s Episode 5 of How AI and ML Will Save Lives in Healthcare, which goes deep on readmissions and care transitions. Here I want to give you the arithmetic first: a proposed prevention funnel that bounds what any program can deliver, the published evidence for each term in it, a simulation showing why ranking patients by benefit beats ranking them by risk, and the sample size you need to prove the program worked. The code runs as written.
The short answer
- Prediction is not the bottleneck. In the standard systematic review of readmission models, C-statistics ran from about 0.55 to 0.75. At 0.70, flagging the top 10% of 20,000 discharges reaches 687 of the 2,800 patients who will be readmitted.
- The funnel shrinks at every step. With 60% of flagged patients engaged and the 18% relative reduction pooled from 42 randomized trials, that program prevents about 74 readmissions a year, 2.6% of the total. A perfect model with an AUC of 1.0 prevents 216.
- High risk is not the same as preventable. About 27% of readmissions are judged potentially preventable. In a randomized trial of an intensive program for the highest-use patients, 62.3% of the intervention group and 61.7% of controls were readmitted within 180 days.
- Rank by benefit, not risk. In Kaiser Permanente Northern California data, targeting by estimated treatment effect could prevent up to 4,458 readmissions a year versus about 1,246 under risk-based targeting. In my simulation, an uplift model fills the same capacity and prevents about 23% more readmissions than a risk model, averaged over ten runs.
- Most programs cannot prove they worked. A randomized holdout analyzed by intention to treat needs about 2,493 flagged patients per arm to detect the effect above, about two and a half years of flags at a 20,000-discharge hospital.
The prevention funnel: one equation for every readmissions program
Every readmissions program, whatever vendor or care model sits behind it, can be written as a chain of shares. I’ll call it the prevention funnel. It is a proposed model, not an established one, but each term in it is something you can measure in your own data.
Prevented readmissions = D × f × PPV × e × ρ, with ρ ≤ p
D is discharges a year. f is the share of discharges you flag, which in practice is set by how many patients the transitions team can carry, not by the model. PPV is the positive predictive value at that flag rate: the share of flagged patients who would be readmitted with no program. e is engagement, the share of flagged patients who actually receive the intervention (reached by phone, seen at home, kept the follow-up visit). ρ is the relative risk reduction the intervention delivers for patients who receive it.
One constraint matters more than the rest. ρ cannot exceed the share of those readmissions that are preventable at all, which I’ll call p. A program cannot avoid a readmission for a cancer progression, a planned procedure or a new fall unrelated to the discharge. So ρ ≤ p, always.
Now put in numbers for a mid-sized system. Take D = 20,000 discharges and a 14% readmission rate, close to the all-payer national rate of 13.9 per 100 admissions in the most recent AHRQ statistical brief. That is 2,800 readmissions a year. The transitions team can take the top 10% of scores, so f = 0.10 and 2,000 patients are flagged. The model has a C-statistic (AUC) of 0.70, typical of published readmission models. Engagement is 60%, and ρ is 0.18, the pooled effect from a meta-analysis of 42 randomized trials of readmission interventions.
The PPV at a 10% flag rate with an AUC of 0.70 works out to 0.344 under a standard binormal assumption (the code is below). The funnel then reads:
- 2,000 patients flagged
- 687 of them would have been readmitted (PPV 0.344)
- 412 of those 687 actually receive the program (60% engagement)
- 74 readmissions prevented (18% relative reduction)
Seventy-four out of 2,800 is a 2.6% drop in total readmissions. That is real and probably worth the money, as the capacity section shows. It is also far too small to see in a monthly readmission rate, which moves more than that from noise alone. That is the first reason most programs look like they did nothing: the honest expected effect is smaller than the variation in the dashboard everyone watches.
The second reason is visible when you vary each term. Raising AUC from 0.70 to 0.80, a large gain that most teams never get, moves prevented readmissions from 74 to 107. A perfect model, with an AUC of 1.0 and a PPV of 1.0, prevents 216, or 7.7%. Call it the super intelligence ceiling: even a model that knew the future exactly would leave more than 92% of readmissions in place, because the other terms in the funnel do not care how good the prediction is. Push everything to its limit (perfect prediction, 100% engagement and an intervention that prevents every preventable readmission, ρ = p = 0.27) and the ceiling is 540, about 19%.
The practical consequence is that the model is the cheapest term to improve and the least valuable one. Engagement, effect size and the choice of whom to treat each matter more, and none of them shows up in the model’s validation report.
readmit_funnel.py
Proposed prevention-funnel model for a readmission program.
prevented = D * f * PPV(AUC, base_rate, f) * e * rho
D discharges per year
f share of discharges flagged (set by care team capacity)
PPV share of flagged patients who would be readmitted without help
e share of flagged patients who actually receive the intervention
rho relative risk reduction among treated patients (bounded by the
preventable share of their readmissions)
PPV comes from a binormal model: scores of readmitted and non-readmitted
patients are normal with equal variance, separated by d = sqrt(2) * Phi^-1(AUC).
"""
from __future__ import annotations
import numpy as np
from scipy.optimize import brentq
from scipy.stats import norm
def ppv_at_flag_rate(auc: float, base_rate: float, flag_rate: float) -> float:
"""PPV when the top `flag_rate` share of patients by score is flagged."""
if auc >= 0.9999:
return min(1.0, base_rate / flag_rate)
d = np.sqrt(2.0) * norm.ppf(auc)
def share_flagged(t: float) -> float:
tpr = norm.sf(t - d) # readmitted patients score ~ N(d, 1)
fpr = norm.sf(t) # everyone else scores ~ N(0, 1)
return base_rate * tpr + (1 - base_rate) * fpr - flag_rate
t = brentq(share_flagged, -10, 10)
tpr = norm.sf(t - d)
return base_rate * tpr / flag_rate
def prevented(discharges: int, base_rate: float, auc: float, flag_rate: float,
engagement: float, rho: float) -> dict:
ppv = ppv_at_flag_rate(auc, base_rate, flag_rate)
flagged = discharges * flag_rate
true_pos = flagged * ppv
treated_true_pos = true_pos * engagement
avoided = treated_true_pos * rho
readmits = discharges * base_rate
return {
"auc": auc,
"ppv": round(float(ppv), 3),
"flagged": round(flagged),
"would_readmit_among_flagged": round(true_pos),
"treated_and_would_readmit": round(treated_true_pos),
"prevented": round(avoided),
"share_of_all_readmissions": round(float(avoided / readmits), 4),
}
if __name__ == "__main__":
D, BASE, FLAG, ENGAGE, RHO = 20_000, 0.14, 0.10, 0.60, 0.18
print(f"Hospital: {D:,} discharges, {D * BASE:,.0f} readmissions a year\n")
for auc in (0.65, 0.70, 0.75, 0.80, 1.00):
print(prevented(D, BASE, auc, FLAG, ENGAGE, RHO))
# Ceiling: perfect prediction, everyone engaged, every preventable
# readmission (27%) actually prevented.
print("\nCeiling:", prevented(D, BASE, 1.00, FLAG, 1.00, 0.27))The binormal assumption (both groups’ scores normally distributed with equal spread) is the standard way to turn an AUC into an operating point when you don’t have the score distributions. With your own model, replace ppv_at_flag_rate with the empirical PPV of the top 10% of last year's scores. At AUCs of 0.65, 0.70, 0.75, 0.80 and 1.0 the script prints 61, 74, 89, 107 and 216 prevented readmissions.
Prediction is the easy part
The readmission prediction model gets most of the attention because it is the part that looks like data science. It is also the part with the most published evidence, and that evidence has been consistent for a long time.
The reference point is Kansagara and colleagues’ systematic review in JAMA, which screened 7,843 citations and found 26 unique readmission models across 30 studies. The nine large US models built on administrative data discriminated poorly, with C-statistics of 0.55 to 0.65. Models that could score patients early in the stay, which is when a transitions team needs the flag, ran from 0.56 to 0.72. The authors concluded that most models performed poorly for both clinical and comparative use. Newer models built on full EHR data, including the vendor scores most hospitals now run, generally report somewhat better discrimination, and external validations tend to land lower than the development papers.
The reason the ceiling is low is not a lack of clever modeling. A 30-day readmission depends heavily on what happens after discharge: whether a prescription gets filled, whether someone notices swelling in the ankles, whether the patient falls on day 12. None of that is in the chart at discharge. A model can see the patient’s burden of illness and history of use very well, and it can see almost nothing of the next 30 days.
The operating point matters more than the AUC. Here is what the funnel script returns at a 10% flag rate for the 20,000-discharge hospital, with engagement at 60% and ρ at 0.18:
Each five-point gain in AUC buys roughly 15 more prevented readmissions a year at this hospital. That is worth having, but it is not where a stalled program is losing most of its benefit. If your team is debating a model replacement to fix a flat readmission rate, run the funnel first: the AUC row you are on is almost never the binding constraint.
One more caution about the score itself. A readmission model trained on outcomes that already include the effect of your current transitions program learns to rate successfully managed patients as low risk. Retrain it after a successful program year and it will steer the team away from exactly the patients the program was helping. That feedback loop is another reason to separate the question of risk from the question of benefit, which is where the next two sections go.
Risk is not preventability
The term that limits everything is p, the preventable share. Two bodies of evidence put it in the same place.
Van Walraven and colleagues reviewed 34 studies that judged whether readmissions were avoidable. The median share was 27.1%, with a range from 5% to 79% depending on how avoidability was defined. Auerbach and colleagues then did the most careful version of the exercise: physicians at 12 US academic medical centers reviewed 1,000 general medicine readmissions, interviewed patients and clinicians, and judged 26.9% potentially preventable. Roughly three in four readmissions would happen anyway.
The causes Auerbach’s team found are worth reading closely, because most of them are not properties of the patient. The factors most strongly tied to a preventable readmission were an emergency department decision to admit a patient who may not have needed a bed, failure to send important information to the outpatient clinicians, discharge before the patient was ready, and no goals-of-care conversation for seriously ill patients. Those are hospital process failures. A model that scores patient characteristics cannot see them, and a phone call after discharge cannot fix most of them.
The intervention side tells the same story. Leppin and colleagues pooled 42 randomized trials of interventions to prevent 30-day readmissions and found a relative risk of 0.82 (95% CI 0.73 to 0.91), the 18% reduction used in the funnel. Interventions with many components were about 1.4 times as effective as simpler ones, and programs that involved more people in care delivery or built up patients’ capacity for self-care were each about 1.3 times as effective. The best-known single trial, Project RED, randomized 738 patients at Boston Medical Center and reported roughly 30% less hospital use (readmissions plus emergency visits) within 30 days. These programs work. They work modestly, on average, and they work through the preventable fraction, not around it.
The sharpest evidence that high risk and high benefit are different things comes from the Camden Coalition’s hotspotting trial, published in the New England Journal of Medicine in 2020. The program enrolled patients with very high utilization, about as high-risk a population as exists, and gave them intensive, well-regarded care management. Finkelstein and colleagues randomized 800 patients. Within 180 days, 62.3% of the intervention group and 61.7% of the control group were readmitted. Earlier before-and-after reports had shown roughly a 40% drop in admissions among participants, and the randomized controls showed the same drop with no program at all. That is regression to the mean: patients selected at a moment of extreme use tend to use less afterward whatever you do.
The lesson for anyone building a readmission program is uncomfortable. The patients at the very top of a risk score are often the ones whose readmissions are least preventable and whose apparent improvement is most likely to be regression to the mean. A model that sorts by risk sends your scarcest resource, nurse and pharmacist time, to the place where it does the least. The fix is to sort by something else.
Target the uplift, not the risk
The quantity a transitions team should rank patients by is uplift: the drop in a patient’s readmission probability if they get the program. Write it as τ(x) = P(readmit | x, no program) − P(readmit | x, program). A risk model estimates only the first term. A patient with 60% risk and nothing modifiable has τ near zero. A patient with 25% risk driven by a missing follow-up appointment and an unfilled prescription may have τ of 0.08 or more.
This is not a theoretical distinction. Marafino and colleagues applied causal machine learning to 1,539,285 index hospitalizations at 21 Kaiser Permanente Northern California hospitals, before and after the system rolled out a readmissions prevention program. They found that treatment effects varied across patients and did not simply track risk. The program as targeted prevented an estimated 1,246 readmissions a year (95% CI 1,110 to 1,381). Targeting by estimated treatment effect instead could have prevented up to 4,458 (95% CI 3,925 to 4,990), with the same kind of program. That is the gap between ranking by risk and ranking by benefit in one of the largest integrated systems in the country.
To see the mechanism in a form you can run, here is a simulation. It creates patients whose risk is driven mostly by comorbidity burden and prior use, while the modifiable share of their risk is driven mostly by follow-up gaps and home support, and is lower for the sickest patients. I calibrated it to the evidence above: a 14.5% readmission rate, a mean modifiable share of 27.2%, and a 19.6% relative reduction if everyone were treated. It then trains a risk model and a T-learner uplift model on two years of history in which the program was offered at random to half of eligible discharges, and asks each to fill the same 10% capacity next year.
"""uplift_vs_risk.py
Simulated comparison of two ways to fill a fixed transitional-care capacity:
rank patients by predicted readmission risk, or by predicted benefit
(uplift: the drop in readmission probability if they get the program).
The data are synthetic. Risk and benefit come from different patient traits,
the pattern the KPNC causal ML study reported: the highest-risk patients are
often not the ones the program helps most.
"""
from __future__ import annotations
import numpy as np
from sklearn.ensemble import HistGradientBoostingClassifier
RNG = np.random.default_rng(7)
def simulate(n: int, rng: np.random.Generator) -> dict:
# Five standardized patient traits.
burden = rng.normal(size=n) # comorbidity and frailty: raises risk, hard to modify
prior_use = rng.normal(size=n) # admissions in the last year: raises risk
followup = rng.normal(size=n) # gaps in follow-up and medication access: modifiable
support = rng.normal(size=n) # caregiver and home support (higher is better)
noise = rng.normal(size=n) # a trait with no effect
X = np.column_stack([burden, prior_use, followup, support, noise])
logit0 = -2.05 + 0.65 * burden + 0.45 * prior_use + 0.35 * followup - 0.25 * support
p0 = 1.0 / (1.0 + np.exp(-logit0)) # risk without the program
# Share of this patient's readmission risk the program can remove.
modifiable = 1.0 / (1.0 + np.exp(-(-1.35 + 1.1 * followup - 0.6 * support - 0.7 * burden)))
efficacy = 0.70 # how much of the modifiable risk the program removes
tau = p0 * modifiable * efficacy # absolute benefit (uplift)
p1 = p0 - tau
return {"X": X, "p0": p0, "p1": p1, "tau": tau, "modifiable": modifiable}
def fit_t_learner(X, treated, y):
"""Two outcome models, one per arm; uplift = p(control) - p(treated)."""
params = dict(max_iter=300, learning_rate=0.05, max_leaf_nodes=15,
min_samples_leaf=200, random_state=0)
m0 = HistGradientBoostingClassifier(**params).fit(X[treated == 0], y[treated == 0])
m1 = HistGradientBoostingClassifier(**params).fit(X[treated == 1], y[treated == 1])
return m0, m1
def prevented_if_targeted(score, tau, k, engagement):
top = np.argsort(-score)[:k]
return tau[top].sum() * engagement
if __name__ == "__main__":
ENGAGE, FLAG = 0.60, 0.10
# Two years of history in which the program was offered at random to half
# of eligible discharges (a randomized holdout). Without randomization the
# T-learner below would pick up selection bias, not benefit.
hist = simulate(80_000, RNG)
treated = RNG.integers(0, 2, size=80_000)
p_obs = np.where(treated == 1, hist["p1"], hist["p0"])
y = RNG.binomial(1, p_obs)
m0, m1 = fit_t_learner(hist["X"], treated, y)
# Next year's 20,000 discharges.
new = simulate(20_000, RNG)
k = int(FLAG * 20_000)
risk_hat = m0.predict_proba(new["X"])[:, 1]
uplift_hat = risk_hat - m1.predict_proba(new["X"])[:, 1]
base = new["p0"].mean()
print(f"Readmission rate without program: {base:.3f} "
f"({new['p0'].sum():,.0f} expected readmissions)")
print(f"Mean modifiable share, all patients: {new['modifiable'].mean():.3f}")
print(f"Relative risk reduction if everyone treated: {new['tau'].sum() / new['p0'].sum():.3f}\n")
rows = {
"Random 10%": RNG.random(20_000),
"Risk model (control-arm fit)": risk_hat,
"Uplift model (T-learner)": uplift_hat,
"Oracle risk": new["p0"],
"Oracle uplift": new["tau"],
}
for name, score in rows.items():
top = np.argsort(-score)[:k]
prev = prevented_if_targeted(score, new["tau"], k, ENGAGE)
print(f"{name:30s} mean risk of flagged {new['p0'][top].mean():.3f} "
f"prevented {prev:6.1f}")The output, run as written:
Three things stand out. First, the risk model is nearly as good as knowing true risk (95.1 versus 96.6), which is the prediction-is-easy point again: a perfect risk score would gain almost nothing here. Second, the uplift model flags patients with a lower average risk (0.294 against 0.384) and prevents about 20% more readmissions with the same staff. Across ten random seeds the gain ran from 20% to 28%, with a mean of 23%, and the risk model’s AUC on new patients averaged 0.72. Third, the oracle uplift row shows the headroom: knowing true benefit would prevent about 47% more than ranking by risk.
The gain in my simulation is smaller than the Kaiser estimate, and it should be. The size of the gain depends on how differently risk and benefit are distributed in your population, which is an empirical question about your patients. The simulation shows the mechanism; your own randomized data tells you the size.
Two cautions about the code. A T-learner fits separate outcome models for treated and untreated patients and subtracts them. It is simple and transparent, and it is noisy, because the difference of two estimates carries the error of both. Causal forests (the grf package in R, or EconML’s CausalForestDML in Python) and doubly robust learners usually do better with real data. More important, every one of these methods needs variation in who received the program that is unrelated to outcomes. Train an uplift model on a history where nurses chose whom to call and it will learn nurse judgment, not benefit. That is why the randomized holdout in the next two sections is not optional: it is both the proof that the program works and the training data for targeting it.
Capacity sets the threshold
The flag rate f is usually chosen by default: the vendor ships a threshold, or the team takes “high risk” as defined by the score’s documentation. It should be chosen from staffing and the marginal number needed to treat, the number of additional patients you must enroll to prevent one more readmission as you move down the list.
Here is the 20,000-discharge hospital again at an AUC of 0.70, 60% engagement and ρ = 0.18. The staffing inputs are assumptions to replace with your own: 3 hours of nurse or pharmacist time per engaged patient (discharge teaching, medication reconciliation, a call within 48 hours, follow-up scheduling), 1,800 productive hours per full-time employee a year, and $130,000 loaded cost per FTE.
The marginal NNT climbs steadily, from 14 for the first 600 enrollees to about 30 for the patients between the 20% and 30% cutoffs, because each step down the list adds patients with lower risk. At these assumptions, though, even the marginal patient costs well under the $16,300 average cost of a readmission that AHRQ reported for 2020. On a whole-system view, cost per prevented readmission is not what binds. Two other things do.
The first is who keeps the savings. Under fee-for-service, a prevented readmission is lost revenue for the hospital. The financial case comes from Medicare’s Hospital Readmissions Reduction Program penalty, from risk contracts, or from both. The HRRP covers six conditions and procedures (heart attack, COPD, heart failure, pneumonia, coronary bypass surgery, and elective hip and knee replacement) and caps the penalty at 3% of base payments. For fiscal 2027, which began October 1, 2026, CMS penalized 2,334 hospitals, about 80% of those in the program, but only 10 at the maximum, and most penalties are below 1%. Fiscal 2027 also brought two changes: excess readmission ratios now count Medicare Advantage patients alongside fee-for-service, and the performance period shrank from three years to two (July 2023 through June 2025), while the penalty itself still applies only to fee-for-service payments. For a system with a large Medicare Advantage or ACO population, the case for spending further down the list is much stronger than for one paid mostly fee-for-service.
The second is hiring. Six FTEs of experienced transitions nurses are much harder to find than $780,000. That is the real argument for ranking by uplift: it does not change the budget, it changes what the budget buys. In the simulation, the risk model has to flag 13% of discharges, 2,600 patients instead of 2,000, to match what the uplift model prevents at 10%. At the staffing assumptions above, that is 30% more nurse time for the same result.
"""capacity_math.py
Staffing, cost and the marginal number needed to treat (NNT) as the flag rate
grows. Uses ppv_at_flag_rate from readmit_funnel.py.
Staffing inputs are assumptions to replace with your own: hours of nurse or
pharmacist time per engaged patient, productive hours per FTE a year, and
loaded cost per FTE.
"""
from readmit_funnel import ppv_at_flag_rate
D, BASE, AUC, ENGAGE, RHO = 20_000, 0.14, 0.70, 0.60, 0.18
HOURS_PER_PATIENT, HOURS_PER_FTE, COST_PER_FTE = 3.0, 1_800, 130_000
COST_PER_READMISSION = 16_300 # AHRQ HCUP average readmission cost, 2020
prev_enrolled, prev_prevented = 0.0, 0.0
print(f"{'flag':>5} {'enrolled':>9} {'FTE':>5} {'cost':>10} {'prevented':>10} "
f"{'NNT':>6} {'marginal NNT':>13} {'cost/prevented':>15}")
for f in (0.05, 0.10, 0.15, 0.20, 0.30):
enrolled = D * f * ENGAGE
prevented = enrolled * ppv_at_flag_rate(AUC, BASE, f) * RHO
fte = enrolled * HOURS_PER_PATIENT / HOURS_PER_FTE
cost = fte * COST_PER_FTE
marginal = (enrolled - prev_enrolled) / (prevented - prev_prevented)
print(f"{f:5.0%} {enrolled:9,.0f} {fte:5.2f} {cost:10,.0f} {prevented:10.1f} "
f"{enrolled / prevented:6.1f} {marginal:13.1f} {cost / prevented:15,.0f}")
prev_enrolled, prev_prevented = enrolled, prevented
print(f"\nAverage cost of one readmission (HCUP 2020): ${COST_PER_READMISSION:,}")Proving it worked
If the expected effect is a 2.6% drop in total readmissions, a before-and-after chart of the hospital’s readmission rate cannot show it. At 20,000 discharges a year, a single month has about 1,667 discharges, and the month-to-month standard error of a 14% rate is about 0.85 percentage points. The program’s whole effect is about 0.37 points. The noise is more than twice the signal, before any secular trend, case-mix change or coding change.
Those last two are not hypothetical. Ody and colleagues, writing in Health Affairs in 2019, found that a change in hospital coding standards that raised patients’ recorded severity explained about half of the decline in risk-adjusted readmissions that had been credited to the HRRP. Readmissions can also fall on paper when returning patients are kept in observation status or treated and released from the emergency department instead of being admitted. And readmission is not the outcome patients care about most. Wadhera and colleagues, studying 8.3 million Medicare hospitalizations in JAMA in 2018, found the HRRP’s implementation was associated with higher 30-day post-discharge mortality for heart failure and pneumonia, though not for heart attack. The association is debated, but the point for program design is not: a readmissions program should be judged on a composite of readmission, observation stay, emergency revisit and death, with mortality as a balancing measure.
The clean answer is the same tool that trains an uplift model: a randomized holdout. When a patient is flagged, randomize whether the transitions team is told. The Camden Coalition trial randomized patients at enrollment, and that design is the only reason we know hotspotting did not reduce readmissions. The analysis is by intention to treat, comparing everyone randomized to the program with everyone randomized to usual care, whether or not they engaged.
That dilution sets the sample size. With a flagged readmission rate of 0.344, 60% engagement and an 18% reduction among those engaged, the program arm’s rate is 0.344 × (1 − 0.6 × 0.18) = 0.307. The standard two-proportion formula gives the patients needed per arm:
n = (z(1−α/2) + z(1−β))² × [p₁(1 − p₁) + p₂(1 − p₂)] / (p₁ − p₂)²
With a two-sided α of 0.05 and 80% power, z values of 1.96 and 0.84, p₁ = 0.344 and p₂ = 0.307, n comes to 2,493 per arm, 4,986 flagged patients in all. If everyone engaged, 877 per arm would do. A single 20,000-discharge hospital flagging 2,000 patients a year needs about two and a half years at a 50% holdout, which is why most single-site programs never get a clean answer. Three ways to get there faster: pool the holdout across hospitals in the system; hold out a smaller share (20% of flags) for longer but start now; or randomize at the unit or week level in a stepped design.
"""holdout_power.py
How many flagged patients a randomized holdout needs to detect a
readmission program's effect, analyzed by intention to treat.
Patients are randomized when flagged, before anyone knows whether they will
engage, so the effect being measured is diluted by the engagement rate:
p_ctrl = PPV (flagged patients' readmission rate)
p_trt = PPV * (1 - e * rho) (only engaged patients get the benefit)
Two-proportion sample size, two-sided alpha, given power:
n per arm = (z_a + z_b)^2 * [p1(1 - p1) + p2(1 - p2)] / (p1 - p2)^2
"""
from math import ceil
from scipy.stats import norm
def n_per_arm(p1: float, p2: float, alpha: float = 0.05, power: float = 0.80) -> int:
z_a = norm.ppf(1 - alpha / 2)
z_b = norm.ppf(power)
return ceil((z_a + z_b) ** 2 * (p1 * (1 - p1) + p2 * (1 - p2)) / (p1 - p2) ** 2)
if __name__ == "__main__":
PPV, RHO = 0.344, 0.18
for e in (1.0, 0.6):
p_trt = PPV * (1 - e * RHO)
n = n_per_arm(PPV, p_trt)
print(f"engagement {e:.0%}: control {PPV:.3f}, program arm {p_trt:.4f}, "
f"difference {PPV - p_trt:.4f}, n per arm {n:,}, total {2 * n:,}")A holdout is sometimes refused on ethical grounds: how can we withhold help from high-risk patients? The honest reply is that the help is unproven for any given population, the team cannot reach everyone flagged anyway, and randomizing among patients the team would not otherwise have reached withholds nothing. Most programs already ration by capacity. A holdout rations by coin flip and learns from it.
Where it breaks
The funnel multiplies its terms as if they were independent. They are not. Engagement is usually lower among the highest-risk patients (harder to reach, more likely to be discharged to a facility), and the effect of the program varies with risk, which is the whole point of the uplift section. Treat the funnel as a bound and a diagnostic, not a forecast. Its job is to show which term is smallest in your program, so you know where effort pays.
The 18% effect is a pooled average across 42 trials of very different interventions, mostly run by motivated teams under study conditions. A program bolted onto an existing case management department may do worse. The 27% preventable share comes from physician review, which is subjective; van Walraven’s range of 5% to 79% shows how much definitions matter. I used both numbers because they are the best published estimates, not because your hospital will match them.
The binormal conversion from AUC to PPV is an approximation. Real score distributions are skewed, and the PPV in the top decile can be higher or lower than the formula says. Use your own scores when you have them.
The uplift simulation is synthetic. I built it so that risk and benefit come from partly different traits, which is what the Kaiser analysis found, but the size of the gain from uplift targeting depends entirely on how much those diverge in your population. If benefit rises in proportion to risk, uplift targeting and risk targeting pick the same patients, and the gain disappears. Uplift models are also harder to validate than risk models: there is no observed label for an individual’s treatment effect, so evaluation relies on held-out randomized data and metrics such as Qini curves.
The Kaiser estimate of up to 4,458 prevented readmissions a year comes from observational data around a program rollout, and the authors present it as what alternative targeting could achieve, not as a measured result. Finally, nothing here addresses the causes in Auerbach’s review that sit inside the hospital: the emergency department admission decision, premature discharge, missing handoffs to outpatient clinicians. A discharge model cannot fix a discharge process. Some of the largest gains available to a population health leader come from process changes that no model will ever flag.
What to do this quarter
None of this argues for dropping your readmission prediction model. It argues for using it as one input to a program designed around the funnel.
Start by measuring your own funnel. Pull last year’s flags and fill in each term: discharges, flag rate, PPV in the flagged group, the share of flagged patients the team actually reached, and the readmission rate of reached versus unreached flagged patients (a biased comparison, but a start). The smallest term is where your next dollar goes. As the funnel table shows, it is rarely the AUC.
Then set the flag rate from capacity and marginal NNT, not from the vendor’s default threshold. Write down the staffing assumption, the cost per enrollee and the payer mix that decides who keeps the savings, so the threshold is a decision someone owns.
Next, turn on a randomized holdout among flagged patients, even a small one. A 20% holdout started this quarter is the only way you will know in two years whether the program works, and it builds the data an uplift model needs. Define the outcome as a composite of readmission, observation stay, emergency revisit and death within 30 days, so the program cannot win by shifting patients between categories.
Finally, take the process causes seriously. Review a sample of your own preventable readmissions the way Auerbach’s team did. If the emergency department admit decision and missing handoffs show up as often as they did in that study, the fix belongs in the discharge workflow, and the model’s job is only to say which patients to start with.
Keep reading. This piece is the companion to the first four episodes of How AI and ML Will Save Lives in Healthcare: Episode 1, sepsis early warning, Episode 2, ward deterioration, Episode 3, stroke detection and Episode 4, hospital capacity and patient flow. Episode 5, on readmissions and care transitions, lands Thursday, November 5, and goes deeper on which transitions interventions hold up in trials and where machine learning helps beyond the risk score. Subscribe on Substack to get it when it publishes.
Brian Curry writes about applied machine learning, causal inference and forecasting for healthcare, energy, commodities and marketing at Vector1 Research in Kansas City.
Sources
- Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA, 2011;306(15):1688–1698.
- van Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ, 2011;183(7):E391-E402.
- Auerbach AD, Kripalani S, Vasilevskis EE, et al. Preventability and causes of readmissions in a national cohort of general medicine patients. JAMA Internal Medicine, 2016;176(4):484–493.
- Leppin AL, Gionfriddo MR, Kessler M, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Internal Medicine, 2014;174(7):1095–1107.
- Boston University Medical Center. Summary of the Project RED trial (summary of Jack BW et al., Annals of Internal Medicine, 2009). ScienceDaily, 2009.
- Finkelstein A, Zhou A, Taubman S, Doyle J. Health care hotspotting: a randomized, controlled trial. New England Journal of Medicine, 2020. Summary: MIT News.
- Marafino BJ, Schuler A, Liu VX, Escobar GJ, Baiocchi M. Causal machine learning study of preventable readmissions at KPNC. Health Services Research, 2020;55(6). Preprint, A Causal Machine Learning Framework for Predicting Preventable Hospital Readmissions: arXiv 2005.14409.
- Ody C, Msall L, Dafny LS, Grabowski DC, Cutler DM. Decreases in readmissions credited to Medicare’s program to reduce hospital readmissions have been overstated. Health Affairs, 2019. Summary: Harvard Medical School, Department of Health Care Policy.
- Wadhera RK, Joynt Maddox KE, Wasfy JH, et al. Association of the Hospital Readmissions Reduction Program with mortality among Medicare beneficiaries hospitalized for heart failure, acute myocardial infarction, and pneumonia. JAMA, 2018;320(24):2542–2552.
- AHRQ Healthcare Cost and Utilization Project. Statistical Brief #304: Characteristics of 30-day all-cause hospital readmissions, 2016–2020. 2023.
- Centers for Medicare & Medicaid Services. FY 2027 Hospital Readmissions Reduction Program fact sheet. 2026.
- Healthcare Dive. Coverage of FY2027 HRRP penalties. October 8, 2026.
Method note: the prevention funnel, the uplift simulation and the capacity table are proposed models with synthetic or assumed inputs, labeled as such in the text. All worked numbers were recomputed in Python from the scripts shown.
