AI Can Prioritize Emergency Department Patients Requiring Urgent Treatment

This research has shown that AI can match the performance of physicians in determining which patients should be seen first.

In this study, researchers at UC San Francisco (San Francisco, CA, USA) utilized anonymized data from 251,000 adult emergency department (ED) visits to test the effectiveness of an AI model. This AI was tasked with extracting and interpreting symptoms from clinical notes to assess the immediacy of patients' treatment needs. The AI's assessments were then compared to the Emergency Severity Index—a 1-5 scale used by ED nurses to triage incoming patients according to the urgency of their conditions. For privacy, the data used were de-identified. The AI technology employed was the ChatGPT-4 large language model (LLM), accessed through UCSF's secure generative AI platform, equipped with extensive privacy measures. To evaluate the AI, researchers used a set of 10,000 matched pairs, totaling 20,000 patients, where each pair consisted of one patient with a severe condition like a stroke and another with a less critical issue such as a broken wrist.

The AI was successful in identifying the more severely ill patient in each pair 89% of the time based solely on symptom data. A focused comparison in a smaller subset of 500 pairs, which also involved physician evaluation, showed the AI's accuracy at 88%, slightly higher than the physician's 86%. Integrating AI into the triage process could potentially alleviate the burden on physicians, allowing them to concentrate on treating the most critical cases and providing a supportive decision-making tool for clinicians handling multiple urgent cases simultaneously. This study stands out as it is among the few that test an LLM with real-world clinical data instead of simulations and is the first to use data from over 1,000 clinical cases and to focus on emergency department visits, where patients present a wide range of medical issues.

“Imagine two patients who need to be transported to the hospital but there is only one ambulance. Or a physician is on call and there are three people paging her at the same time, and she has to determine who to respond to first,” said lead author Christopher Williams. “Upcoming work will address how best to deploy this technology in a clinical setting.”

Related Links:
UC San Francisco