AI Model Combines Chest X-rays with Patient Data to Improve Diagnosis

While existing artificial intelligence (AI)--based solutions are usually designed to work with just one form of data, transformer-based neural networks offer the capability to merge both kinds for more precise diagnoses. Originally intended for processing human language, these transformer models have inspired sizable language models like ChatGPT and Google's Bard. Unlike their convolutional neural network counterparts, which focus mainly on imaging data, transformer networks have a more general application. They employ what's known as an attention mechanism, allowing the network to understand relationships within its input data. This feature is particularly well-suited for medical fields where diagnosis often involves combining varied data types like patient information and imaging results. Now, a newly developed transformer AI model combines clinical patient data with imaging information to enhance diagnostic accuracy on chest X-rays.

Researchers at University Hospital Aachen (Aachen, Germany) have developed a transformer model specifically for medical use. They trained the system using both imaging and non-imaging data from two databases, which together include records of over 82,000 patients. This model has been taught to identify up to 25 different conditions using either type of data or a mix of the two, known as multimodal data. When compared to other existing models, this new multimodal approach demonstrated a better diagnostic performance across the board. According to the research team, this model not only has the potential to assist healthcare providers who are facing increasing workloads but could also serve as a blueprint for seamlessly integrating large sets of data.

"With patient data volumes increasing steadily over the years and time that the doctors can spend per patient being limited, it might become increasingly challenging for clinicians to interpret all available information effectively," said study lead author Firas Khader, M.Sc., a Ph.D. student at University Hospital Aachen. "Multimodal models hold the promise to assist clinicians in their diagnosis by facilitating the aggregation of the available data into an accurate diagnosis."

Related Links:
University Hospital Aachen