Clinical System One: Streamlining Patient Record Decision-Making

Paper: A general-purpose System One decision model for clinical records

Authors: Andrew Zhang, Trinh T. L. Vuong, Sophia J. Wagner, Dongmin Bang, Tong Ding, Yuchang Su, Mary Cipperman, Christina S. Chen, Judith Lefkes, Ramin Kahidi, Ming Y. Lu, Caiwei Tian, Tianhao Luo, Jingnan Ma, Rowland Pettit, Long Phi Le, Faisal Mahmood

Executive Summary

The research introduces Clinical System One (CS1), a decision model designed to improve the efficiency of extracting information from patient records in healthcare settings. With 4 billion parameters, CS1 is tailored to provide quick, reliable answers to a variety of clinical questions without the need for extensive retraining for specific tasks. This capability is particularly significant given the growing demand for efficient data processing in clinical care, health system operations, and biomedical research.

CS1 was developed using a vast dataset comprising 12.4 million questions and answers sourced from 3.6 million patients across a multi-hospital health system. It integrates structured data, observed patient outcomes, and language model supervision, analyzing clinical notes, diagnostic reports, and patient timelines. This approach enables CS1 to deliver multiple answers in a single processing step, each accompanied by a probability score that indicates the certainty of the response. If the model encounters uncertain cases, it can flag these for further human review, enhancing the reliability of the information provided.

In performance evaluations, CS1 demonstrated impressive results. When tested on a dataset of 963,408 patients, it outperformed open decision models with up to 27 billion parameters on 13 out of 14 independent tasks. Moreover, when assessed against 21 external datasets, CS1's performance was comparable to that of the most robust tested model, which had 27 billion parameters, while also achieving 6.4 times faster inference speed with similar workloads.

The research also explored the integration of CS1 with a generalist language model, enhancing its capabilities further. This combined approach was able to generate responses based on complete multi-year patient histories and successfully screened patients against the eligibility criteria for 17 clinical trials. Remarkably, it identified four times more eligible patients among its top 20 candidates compared to traditional keyword search methods, all while consuming 690 times fewer language model tokens than a model that examined every patient record in detail.

These findings suggest a potential division of labor in clinical AI applications. Smaller decision models like CS1 could handle the repetitive task of reading medical records, while larger generative models could focus on formulating questions and synthesizing evidence. This could lead to more efficient workflows in clinical environments, where timely and accurate information is critical for patient care and research advancements.

The implications of this research are significant, particularly as healthcare organizations seek to leverage AI for improved operational efficiency and patient outcomes. By enabling faster and more accurate data retrieval from patient records, CS1 may enhance clinical decision-making and support more effective biomedical research initiatives.

Academic Abstract

Clinical care, health-system operations and biomedical discovery depend on extracting reliable answers from patient records, yet applying diverse clinical questions across entire populations remains computationally expensive. Here we present Clinical System One (CS1), a 4-billion-parameter decision model for fast, typed judgments from clinical records, along with two agentic harnesses in which CS1 works together with a generalist language model. CS1-4B is trained on 12.4 million questions and answers from 3.6 million patients across a multi-hospital health system, combining structured-data keys, observed outcomes and language-model supervision across clinical notes, diagnostic reports and patient timelines. It answers multiple typed questions in a single forward pass without task-specific retraining, and each answer carries a probability that can route uncertain cases to human review. On 963,408 held-out patients, CS1-4B outperformed open decision models up to 27 billion parameters on 13 of 14 tasks scored independently of its training teacher. Across 21 external data sets, its aggregate performance approached that of the strongest tested 27-billion-parameter model, with 6.4-fold faster inference in a matched workload. Integrating CS1-4B with a language model agent produced cited answers over complete multi-year patient histories and screened a patient population against the eligibility criteria of 17 registered clinical trials, placing four times as many judged-eligible patients among its top 20 candidates as keyword search while using 690-fold fewer language-model tokens than a language model reading every record. These findings support a division of labour in clinical AI, with small decision models performing repeated record reading and generative models formulating questions and synthesizing evidence.