New AI Model Based on 3D CT Scans Improves Accuracy of Machine Learning in COVID-19 Diagnosis

An international team of researchers, led by the University of Cambridge (Cambridge, England) and the Huazhong University of Science and Technology (Hubei, China), used a technique called federated learning to build their model. Using federated learning, an AI model in one hospital or country can be independently trained and verified using a dataset from another hospital or country, without data sharing. The researchers based their model on more than 9,000 CT scans from approximately 3,300 patients in 23 hospitals in the UK and China. Their results provide a framework where AI techniques can be made more trustworthy and accurate, especially in areas such as medical diagnosis where privacy is vital.

AI has provided a promising solution for streamlining COVID-19 diagnoses and future public health crises. However, concerns surrounding security and trustworthiness impede the collection of large-scale representative medical data, posing a challenge for training a model that can be used worldwide. In the early days of the COVID-19 pandemic, many AI researchers worked to develop models that could diagnose the disease. However, many of these models were built using low-quality data, ‘Frankenstein’ datasets, and a lack of input from clinicians. Many of the same researchers from the current study highlighted that these earlier models were not fit for clinical use in the spring of 2021.

The international team of researchers used two well-curated external validation datasets of appropriate size to test their model and ensure that it would work well on datasets from different hospitals or countries. The researchers based their framework on three-dimensional CT scans instead of two-dimensional images. CT scans offer a much higher level of detail, resulting in a better model. They used 9,573 CT scans from 3,336 patients collected from 23 hospitals located in China and the UK.

The researchers also had to mitigate for bias caused by the different datasets, and used federated learning to train a better generalized AI model, while preserving the privacy of each data centre in a collaborative setting. For a fair comparison, the researchers validated all the models on the same data, without overlapping with the training data. The team had a panel of radiologists make diagnostic predictions based on the same set of CT scans, and compared the accuracy of the AI models and human professionals. The researchers say their model is useful not just for COVID-19, but for any other diseases that can be diagnosed using a CT scan.

“AI has a lot of limitations when it comes to COVID-19 diagnosis, and we need to carefully screen and curate the data so that we end up with a model that works and is trustworthy,” said co-first author Hanchen Wang from Cambridge’s Department of Engineering.

“Before COVID-19, people didn’t realize just how much data you needed to collect in order to build medical AI applications,” said co-author Dr. Michael Roberts from AstraZeneca and Cambridge’s Department of Applied Mathematics and Theoretical Physics. “Different hospitals, different countries all have their own ways of doing things, so you need the datasets to be as large as possible in order to make something that will be useful to the widest range of clinicians.”

Related Links:
University of Cambridge
Huazhong University of Science and Technology