A model measured where it runs
The Epic Sepsis Model is among the most widely implemented proprietary prediction models in US hospitals. In 2021, a University of Michigan team published an external validation in JAMA Internal Medicine covering 27,697 patients across 38,455 hospitalizations. The area under the curve came in at 0.63, against the 0.76 to 0.83 the vendor had reported in its own documentation. At the recommended threshold the model found 33% of sepsis cases, missed 1,709 of the 2,552 patients who developed sepsis, and alerted on 18% of all hospitalizations.
A second external validation, published in JAMIA Open in December 2024, put the same version through 145,885 encounters at two county emergency departments in Harris County, Texas. Within a six-hour window it reached 14.7% sensitivity, 95.3% specificity and a positive predictive value of 7.6%.
The timing figure is the one to sit with. Median lead time was zero minutes. In half the cases where an alert and sepsis both occurred, the alert fired after onset. The authors concluded that the model provides suboptimal diagnostic characteristics for undifferentiated patients in a county emergency department. A system can be live, integrated and firing on schedule while contributing nothing to the moment it was bought to change.
The second measurement
Epic rebuilt the model and released a second version in 2022. In February 2026, JAMA Network Open published a prospective multicenter validation of it: 227,091 inpatient encounters across Michigan Medicine, Oregon Health & Science University, Emory Healthcare and MetroHealth.
Discrimination improved substantially. The area under the curve ran 0.82 to 0.92 across the four sites, against 0.65 to 0.84 for the first version, and positive predictive value rose to a range of 0.13 to 0.26 from 0.07 to 0.14. Median prediction lead time ran 1.4 to 7.1 hours ahead of clinician recognition. The authors report that the new model outperformed its predecessor across all metrics, at all thresholds, at all sites.
What remains is operational. At a positive predictive value of 0.13, roughly seven alerts fire for each case that materializes, and alert volume concentrates in the hours right after admission. The ten-point spread across four health systems is itself a finding: one model, four sites, four answers, which is why the authors' recommendation to implementing institutions was local validation, workflow design to absorb false positives, and silencing strategies to hold the burden down.
Both measurements came from teams who had no hand in building the model. That is the mechanism worth copying. A vendor figure describes the model; a site figure describes the model inside your case mix, your documentation habits and your definition of the outcome, and the second one is what reaches your patients.
Three ways reliability decays
Finlayson and colleagues, writing in the New England Journal of Medicine in 2021, sorted the causes of dataset shift into three groups. Changes in technology cover new acquisition devices, software updates and infrastructure migrations. Changes in population and setting cover case mix, disease incidence, seasonality and new treatments. Changes in behavior cover clinical incentives, documentation practice, nomenclature revisions, and the adaptations clinicians make in response to the model itself.
That paper carries a concrete example. A sepsis-alerting model at the University of Michigan hospital was deactivated in April 2020 because shifts in patient demographics during the COVID-19 pandemic produced spurious alerting. The model was unchanged. The world it measured moved, and the alerts moved with it.
Decay also arrives quietly. Davis and colleagues built seven acute kidney injury models on a 2003 cohort of Veterans Affairs admissions and validated them across the following nine years, reporting in JAMIA in 2017. Discrimination held for every model while calibration degraded, with the models increasingly overpredicting risk. A team watching only the area under the curve would have seen nine stable years.
What a monitoring setup measures
Inputs come first: the distribution of every feature the model consumes, the rate of missingness in each, and the arrival of anything new — a lab sent by a different vendor, a unit that changed, a code set that was revised. Input monitoring is the layer that can fire before the outputs go wrong, and it works while outcome labels are still pending.
Outputs come second. Score distributions, the alert rate per thousand encounters, and threshold crossings, tracked on a rolling window. Outcomes come third, and they need two separate measurements: discrimination answers whether the model ranks patients correctly, and calibration answers whether a score of 0.30 means thirty in a hundred. The acute kidney injury result is the reason both belong on the dashboard.
Then there is the human layer, which most dashboards omit. A 2026 cross-sectional survey in JMIR Human Factors put 15 questions to 65 radiologists in a national teleradiology practice using an FDA-cleared tool for intracranial hemorrhage detection. 18.5% agreed that false-positive alerts were infrequent enough to be acceptable. 33.8% agreed that the extra time spent reviewing false positives outweighed the benefits, and 10.8% reported any reduction in overall interpretation time.
Those are perceptions, and the authors label them as such. They are also the only numbers on that list that describe what a cleared, deployed tool does to a working day. Override rates, time from alert to action, and the share of alerts anyone acted on belong beside the accuracy metrics, because a model whose alerts are dismissed has stopped producing clinical effect whatever its score reads.
Subgroups carry their own numbers
Aggregate accuracy averages over the people a system fails. The clearest published demonstration remains Obermeyer and colleagues in Science in 2019, examining a commercial algorithm used to enroll patients in high-risk care management. It predicted future health costs as a stand-in for health need, and because less is spent caring for Black patients at equal levels of illness, it scored them as healthier. Correcting that would have raised the share of Black patients receiving the extra help from 17.7% to 46.5%.
Bias also enters through instruments. Sjoding and colleagues reported in the New England Journal of Medicine in 2020 that among patients reading 92% to 96% on pulse oximetry, arterial measurement found oxygen saturation below 88% in 12% of Black patients and 4% of White patients at one site, and in 17% against 6% in a multicenter cohort. Any model consuming SpO2 inherits that error, stratified by race, through no fault of its own.
Subgroup performance is therefore a monitored quantity with its own thresholds, sliced by the attributes that bear on the outcome. It is measured rarely. A 2025 JAMIA survey of 43 US health systems found that 74% always measured uptake of their AI tools while 17% always measured health equity or disparities.
What independent oversight actually exists
A buyer might reasonably assume that independent certification of health AI exists. For roughly two years it looked as though it would. The Coalition for Health AI proposed a national network of independent assurance labs that would test health AI for validity, fairness and safety, and the proposal was widely reported as the coming industry standard. The network never opened. STAT reported the end of the effort in May 2025, and Fierce Healthcare covered the scrapped labs in its own account.
What CHAI has shipped is lighter and genuinely useful. An applied model card standardizes what a developer discloses at procurement — intended use, target population, data types, performance metrics, known risks and biases, maintenance needs — and a public registry collects them. Evaluating a model against a locally representative dataset stays with the buyer.
The surrounding apparatus is voluntary in the same way. The Joint Commission and CHAI published guidance on the responsible use of AI in healthcare in September 2025, organized around seven areas including ongoing quality monitoring, risk and bias assessment, and blinded reporting of AI safety events, with playbooks and a voluntary certification to follow. The STANDING Together recommendations, published in The Lancet Digital Health in January 2025 from a Delphi process of 194 panellists across 25 countries, set 29 items for reporting dataset composition and limitations. Both are guidance, and enforcement machinery is a separate thing entirely.
One mechanism does bind. The FDA finalized guidance in December 2024 on predetermined change control plans for AI-enabled device software functions. A PCCP names the specific modifications a manufacturer intends to make, the protocol for developing and validating each, and an assessment of their impact. Authorized up front, it lets a manufacturer ship those changes without a new marketing submission, provided they stay inside the authorized intended use. It reaches regulated devices, which leaves most predictive models running inside health systems governed by whatever that health system wrote down. A 2025 survey of 233 health systems by HFMA and Eliciting Insights found 88% using AI internally and 18% holding a mature governance structure paired with a formed strategy.
What to write down before go-live
Four things belong in the contract. Notification of any model change, with a defined lead time and a description of what changed. A version identifier logged with every prediction, so a shift in behavior can be traced to a release. Access to the vendor's own monitoring data on your instance. And a stated trigger and method for local re-validation, with the cost settled in advance.
Four things belong in the runbook. The named owner of the model, and the named owner of the decision to suspend it. Review thresholds written as numbers on the metrics above, including alert rate and positive predictive value alongside accuracy. The fallback pathway that runs while the model is off, rehearsed at least once. And a monitoring setup that has been made to fire on purpose, because an alarm that has stayed silent for eighteen months is an untested alarm.
This is the operational discipline any measured system in a hospital already runs on. The case for extending it to a model rests on the model's failure mode, which is silence. A broken infusion pump announces itself. A model that has drifted keeps producing scores that look exactly like the scores it produced while it was right.
What we sign up to
DotCross has had systems in production since 2021, and running one is different work from building it. The build ends. The running does not, and the second job is where a model earns or loses the confidence the first one bought.
So the monitoring plan is written before go-live, alongside the build, and it names things a person can act on: the metrics and their review cadence, the thresholds as actual numbers, the owner of the model, the owner of the decision to suspend it, and the path back to the previous version. Every prediction is logged with the version that produced it, which is what turns "the output feels different this month" into a question with an answer.
The reason this matters more for a model than for the rest of the estate is the failure mode. A broken pump alarms. A degraded model keeps answering in a confident voice, at the same rate, in the same format, and the only thing that changes is how often it is right. Something has to be watching for that, and it has to have been agreed while everyone was still optimistic.
