The work is reading at volume
A systematic review team working a single clinical question screens tens of thousands of titles and abstracts to find the few dozen studies that answer it. Cochrane's own figures give the scale: in 2018 its machine-learning trial classifier removed 185,039 records from manual screening out of 449,480 processed, work that would otherwise have gone to human screeners one abstract at a time.
The job decomposes cleanly. Find the candidate documents, decide which are eligible, extract what each one reports, and assemble the findings into a statement with its evidence attached. Each step has a defined input, a defined output, and a record of what was decided — which is most of what makes a task suitable for an agent.
One further property makes the work unusually tractable. The corpus is bounded and stably identified: every paper has a DOI or a PMID, every guideline a version and a date. An answer here can point at the documents it came from, and a reader can open them. That is the whole basis of any trust placed in the output.
Where the numbers hold up
Screening carries the most measurement. A 2024 JAMA Network Open study by Takehiko Oami and colleagues ran a large language model across 16,669 publications retrieved for five clinical questions in the Japanese sepsis guidelines. Sensitivity was 0.75, rising to 0.89 once the prompt was revised, specificity was 0.99, and the reported screening time was 1.3 minutes against 17.2 for conventional methods.
The trade-off inside that task has since been mapped. A 2025 study from Oxford, in the Journal of the American Medical Informatics Association, tested model ensembles against 119,695 articles retrieved for 23 Cochrane reviews. Ensembles tuned to miss nothing reached perfect sensitivity and cut screening workload by up to 41.8%. The configuration that cut workload by 99.1% recovered 69% of the eligible studies, which disqualifies it for a review that has to be complete.
Trial matching has been measured twice over. TrialGPT, from the NIH's National Library of Medicine, reached 87.3% criterion-level accuracy across 1,015 patient-criterion pairs scored by three physicians, in Nature Communications in November 2024. Mass General Brigham then tested RECTIFIER, a retrieval-grounded tool that reads charts against eligibility criteria and leaves the enrolment decision with a person, as a randomised comparison inside a trial of nearly 4,500 patients. Published in JAMA in 2025, it enrolled at almost double the rate of manual screening, with no significant difference in eligibility or enrolment by race, gender or ethnicity. That second finding matters: screening is where a bias problem surfaces first.
Regulatory writing is earlier, and its evidence is thinner. A September 2025 preprint on AutoIND, which drafts investigational new drug nonclinical summaries, reports first drafts falling from roughly 100 hours to under four across 61 reports, with a blinded regulatory assessor finding no critical errors and quality scores of 69.6% and 77.9%. Elsevier's claim that Embase AI halves biomedical review time comes from its own beta users, and belongs in another column.
Four ways the output goes wrong
Fabrication is the failure everyone names, and its distribution is the useful part. In a JMIR Mental Health study published in November 2025, Jake Linardon and colleagues had GPT-4o write six literature reviews and checked all 176 citations produced. Thirty-five of them, 19.9%, pointed at work that does not exist. The rate was 6% for major depressive disorder and 28% and 29% for binge eating disorder and body dysmorphic disorder. Fabrication concentrates where the literature is thin, which is where a reader can least catch it.
Mis-summarisation is quieter and harder to catch. Uwe Peters and Benjamin Chin-Yee compared 4,900 model-written summaries against their source texts in Royal Society Open Science in April 2025. Several leading models widened the scope of a finding — turning a result in one trial population into a general claim — in 26% to 73% of cases, roughly five times the rate of human summaries of the same work. Asking explicitly for accuracy made overgeneralisation about twice as likely, and newer models did worse than older ones.
Currency is structural, and the model cannot solve it alone. Kaveh Shojania's survival analysis of 100 systematic reviews found a median of 5.5 years before a signal to update appeared, with 23% out of date within two years and 7% on publication. Retraction compounds it. A March 2025 JAMA Internal Medicine study identified 50 systematic reviews whose meta-analyses included retracted trials; removing those trials moved the primary effect estimate by at least 10% in 42% of cases and by at least half in 19%. In 74% of the reviews the retraction landed after publication.
These failures are live in deployed systems. In July 2025 CNN reported that FDA staff using Elsa, the agency's internal generative AI tool, had watched it produce studies that do not exist and misstate real ones, and the agency's head of AI confirmed on the record that it hallucinates. One employee's summary: anything you lack time to double-check is unreliable.
The engineering answer to each
Fabrication has a mechanical answer. The agent retrieves from an index you control, every claim carries an identifier back to the retrieved passage, and that identifier is resolved against the source database before the answer renders. A claim whose citation fails to resolve gets withheld. That turns an open-ended generation problem into a lookup with a pass condition.
The stronger version makes provenance a property of the data structure. VISTA Architect, from a Stanford Medicine group in June 2026, holds patient records in a graph that preserves the original record structure with full provenance attached, and the model extracts against that graph. Across 1,180 thoracic oncology patients and 17,700 variable-level checks it reported 96.4% accuracy (95% CI 96.1–96.7%) on the fifteen variables a tumour board uses, and its authors state that validation outside thoracic oncology is future work. The design is the transferable part: every extracted value knows which record it came from, because the graph carries the link.
Currency and retraction are index problems, and they are solved at the index. Crossref has hosted the Retraction Watch database as an open, daily-updated dataset since 2023, so retraction status can sit on the document record beside publication date and guideline version, with the agent required to surface all three. Evidence surveillance is the same machinery pointed at time: re-run the query on a schedule, and report what changed.
Recall is a calibration decision, and Cochrane's trial classifier shows what taking it seriously looks like. It was tuned to at least 99% recall, which cost precision — 0.08 in bootstrap validation, meaning most of what it passes forward is still ineligible and still gets read by a person. Validation against 44,007 trials in Cochrane Reviews retrieved 43,783 and missed 224; the miss count is published because the number is the point. Human review then earns its cost at extraction: an effect size, a population definition, an inclusion criterion.
What a benchmark score proves
Benchmark scores travel badly. In JAMA Network Open in August 2025, Suhana Bedi and colleagues at Stanford took 100 MedQA questions, replaced the correct option with "none of the other answers", and kept the 68 a clinician confirmed. Accuracy fell for all six models tested, by 8.8 percentage points for the best and 38.2 for the worst. The original scores had been measuring familiarity with the shape of the question.
MedAgentBench, from Stanford and published in NEJM AI, is the closest thing to a rehearsal: 300 physician-written tasks put to an agent inside a FHIR-compliant record system of 100 synthetic patients and 700,000 data elements. Claude 3.5 Sonnet v2 succeeded on 69.67% overall and 85.33% on pure retrieval, GPT-4o on 64.0%. The error analysis is the useful part. Most failures were operational — invalid actions, answers wrapped in prose where a bare number was expected — with one model emitting invalid actions in 54% of cases. Medical knowledge was rarely the binding constraint.
MedHELM, published in Nature Medicine in 2026, widens the aperture: 121 clinical tasks in five categories, built with 29 clinicians. Models score highest on note generation and patient communication, and lowest on clinical decision support. Its language-model jury matched clinician ratings at an ICC of 0.47, against 0.43 between clinicians — a figure that says as much about clinician disagreement as about the jury.
What a benchmark establishes is a floor: a system can do a kind of task on somebody else's corpus, under somebody else's definition of correct. Whether you can use it turns on a different question — whether the result holds on your questions, against your index, judged by people who know the subject.
Evaluating one before you rely on it
Evaluation starts with a reference set built by domain experts before the agent runs: fifty to a few hundred real questions from the work it is meant to support, each with the answer a specialist would give and the sources that carry it. Building this is slow, and everything downstream is scored against it.
Score three things separately, because they fail separately. Is the claim true? Does the cited source say it — the same population, the same outcome, the same magnitude? Is the source current and unretracted? A system can pass the first and fail the second, and that case does the most damage, because the answer reads as sourced.
Traceability sits ahead of all of it as a pass condition. Every claim resolves to a document, the document opens, and the passage supports the sentence. An agent that produces good answers with citations you cannot follow has handed you a research problem. Grade the resolution rate first, and treat anything under 100% as a defect with a cause.
The claim and its route back
The pattern generalises. Evidence synthesis suits an agent because the corpus is closed, the identifiers are stable, and the output can be made checkable end to end. Every measured result above comes from a setup where somebody could go and look.
That is also the boundary. Where an answer resolves to documents a reader can open, its failures become findable and countable — a fabricated citation fails to resolve, a stale one carries its date, an overreach is visible against the abstract it came from. Where the answer arrives without that route back, the reader is left checking the work by doing the work. A claim you cannot trace is a claim you cannot use.
The standard we hold ourselves to
Traceability is an architecture decision, and it is made early or it is made never. Bolting citations onto a system that already answers produces a plausible reference beside a generated sentence, which is the failure mode wearing the costume of the fix. Carrying provenance through the pipeline from the first query is what makes an answer checkable at the end of it.
So an evidence system we build resolves every claim to a document a person can open, keeps the identifier and the retrieval date with the claim, and is scored against a reference set that domain experts built and agreed before the system saw it. Where the consequence is highest, a person signs the output. Those checkpoints are chosen for consequence, and their cost is part of the design.
The standard is simple to state and demanding to meet: an answer that cannot be traced is a draft. It may well be correct. It is unusable as evidence until someone can follow it back, and in clinical and regulatory work the following back is most of the value.
