New AI-Driven Model for Predicting COVID-19 Impact Dramatically Outperforms Other Models

The use of AI allows a model to make predictions based on observations of what is actually happening - for example, increasing cases among specific populations - as opposed to what the model's designers think will happen. With the use of AI, it is possible to discover patterns hidden in data that humans alone might not recognize.

The researchers evaluate the accuracy of the new CS156 model (so-named for the Caltech computer science class where it got its start) by comparing it to the predictions of an ensemble model built by the Centers for Disease Control and Prevention from 45 major models from universities and institutes across the country. Using 1,500 predictions as points of comparison with the CDC ensemble, the researchers found that the CS156 model was more accurate than the ensemble model 58% of the time, as of November 25. It also routinely outperforms the benchmark projections of the Institute for Health Metrics and Evaluation (IHME).

The CS156 model is an amalgam of several models developed the team over the past nine months; the weight that each model has on the overall CS156 model's output is determined based on its performance over the previous weeks. The team is currently expanding the CS156 model based on feedback from public health officials in the hope that it can be a lifesaving tool to guide policy decisions. The team has been contacted by officials from the California Department of Public Health (CDPH) and from representatives of the New York City Commissioner of Health. Based on their feedback, the model is being modified to allow public health officials to predict how various interventions - like mask mandates and safer-at-home orders - affect control of the spread of the disease. Armed with those predictions, public health officials would be better able to evaluate which interventions are most likely to help. CDPH already uses the CS156 model in its decision-making.

"AI is a powerful tool, so it only makes sense to apply it to one of the most urgent problems the world faces," said Yaser Abu-Mostafa (PhD '83), professor of electrical engineering and computer science, who led the development of the new CS156 model. "All of us who are working on COVID-19 models are working toward the same goal: winning the war against the pandemic. We're here to do our part."

Related Links:
California Institute of Technology