We use cookies to understand how you use our site and to improve your experience. This includes personalizing content and advertising. To learn more, click here. By continuing to use our site, you accept our use of cookies. Cookie Policy.

Features Partner Sites Information LinkXpress hp
Sign In
Advertise with Us
IBA-Radcal

Download Mobile App




New Scoring Systems Increase Accuracy of AI-Generated Radiology Reports

By HospiMedica International staff writers
Posted on 07 Aug 2023

Artificial intelligence (AI) tools that efficiently produce detailed narrative reports of CT scans or X-rays can significantly lighten the workload of busy radiologists. More...

These AI reports go beyond simple identification of abnormalities and instead provide complex diagnostic information, detailed descriptions, nuanced findings, and appropriate degrees of uncertainty, similar to how human radiologists describe scan results. While several AI models capable of generating such detailed medical imaging reports have emerged, automated scoring systems meant to assess these tools are proving to be less effective at gauging their performance, according to a new study.

In the study, researchers at Harvard Medical School (Boston, MA, USA) tested various scoring metrics on AI-generated narrative reports and had six human radiologists read these reports. The analysis revealed that automated scoring systems performed poorly compared to human radiologists when it came to evaluating AI-generated reports. These systems misinterpreted and even missed significant clinical errors made by the AI tool. Ensuring the reliability of scoring systems is crucial for AI tools to continue improving and gaining clinicians' trust. However, the metrics tested in the study failed to reliably identify clinical errors in the AI reports, highlighting an urgent need for improvement and the development of high-fidelity scoring systems that accurately monitor tool performance.

In order to create better scoring metrics, the research team designed a new method called RadGraph F1 for evaluating the performance of AI tools generating radiology reports from medical images. Additionally, they created a composite evaluation tool called RadCliQ, which combines multiple metrics to produce a single score that more closely aligns with how a human radiologist would assess an AI model's performance. Using these new scoring tools, the researchers evaluated several state-of-the-art AI models and found a notable gap between their actual scores and the top possible scores.

Going forward, the researchers envision building generalist medical AI models capable of performing various complex tasks, including solving novel problems. Such AI systems could effectively communicate with radiologists and physicians about medical images, assisting in diagnosis and treatment decisions. The team also aims to develop AI assistants that can explain imaging findings directly to patients using everyday language, enhancing patient understanding and engagement. Ultimately, these advancements could revolutionize medical imaging practices, improving efficiency, accuracy, and patient care.

“Accurately evaluating AI systems is the critical first step toward generating radiology reports that are clinically useful and trustworthy,” said study senior author Pranav Rajpurkar, assistant professor of biomedical informatics in the Blavatnik Institute at HMS. “By aligning better with radiologists, our new metrics will accelerate development of AI that integrates seamlessly into the clinical workflow to improve patient care,”

Related Links:
Harvard Medical School 


Gold Member
SARS‑CoV‑2/Flu A/Flu B/RSV Sample-To-Answer Test
SARS‑CoV‑2/Flu A/Flu B/RSV Cartridge (CE-IVD)
Hand-held Hemoglobin Analyzer
DiaSpect Tm
Blood Pressure Monitor
Cuff Blood Pressure Monitor
Wound Irrigation Solution
Prontosan®
Read the full article by registering today, it's FREE! It's Free!
Register now for FREE to HospiMedica.com and get access to news and events that shape the world of Hospital Medicine.
  • Free digital version edition of HospiMedica International sent by email on regular basis
  • Free print version of HospiMedica International magazine (available only outside USA and Canada).
  • Free and unlimited access to back issues of HospiMedica International in digital format
  • Free HospiMedica International Newsletter sent every week containing the latest news
  • Free breaking news sent via email
  • Free access to Events Calendar
  • Free access to LinkXpress new product services
  • REGISTRATION IS FREE AND EASY!
Click here to Register








Channels

Artificial Intelligence

view channel
Image Credit: Shutterstock

Explainable AI Examines Sepsis Mortality Risk and Emergency Care Delays

Preventing sepsis-related deaths in emergency departments (EDs) remains challenging because patient deterioration can result from multiple rapidly evolving and interconnected factors. These include triage... Read more

Patient Care

view channel
Image: A view inside the custom virtual reality environment used in the study: a calm room with a teddy bear, soft colors and large windows looking out over the sea. (Image Credit: Walaa S. Abu Rukbah et al., Virginia Commonwealth University; CC-BY 4.0, https://creativecommons.org/licenses/by/4.0/)

Virtual Reality Program May Reduce Anxiety Before Pediatric Surgery

Presurgery anxiety affects an estimated 50% to 75% of children undergoing surgery and can complicate anesthesia induction, increase postoperative pain, and prolong recovery. Sedation is often used to manage... Read more

Health IT

view channel
Image Credit: Shutterstock

Telemonitoring Program Improves Quality of Life After Heart Failure Discharge

Heart failure is a chronic condition associated with high mortality, frequent hospitalizations, and reduced quality of life. Patients face particularly high risks of readmission and death after discharge,... Read more

Point of Care

view channel
Image: Jessica Ramella-Roman, professor of biomedical engineering at Florida International University, holding the PPRIM (Portable Preterm Imaging) device designed to detect early signs of preterm labor risk. (Photo courtesy of Florida International University)

Handheld Imaging Device May Help Identify Preterm Birth Risk Earlier

Preterm birth, defined as delivery before 37 weeks of pregnancy, can lead to serious neonatal complications, including neurological, developmental, and respiratory problems. However, identifying pregnancies... Read more

Business

view channel
Image: Stryker has acquired ZuriMED, adding the FiberLocker System, a rotator cuff augmentation technology designed to strengthen repairs, to its Sports Medicine portfolio. (Photo courtesy of ZuriMED)

Stryker Completes Acquisition to Broaden Shoulder Surgery Offerings

Stryker (Portage, MI, USA) has completed its acquisition of ZuriMED (Zurich, Switzerland), a privately held developer of the FiberLocker System, expanding its Sports Medicine portfolio with a commercialized... Read more
Copyright © 2000-2026 Globetech Media. All rights reserved.