Surgical Risk Calculator Results Proved Valid

The researchers evaluated model performance for the development (60% of records), validation (20%), and test (20%) datasets.

Prediction equations from the development dataset were recalibrated using restricted cubic splines estimated from the validation dataset. The researchers then analyzed the calibration of the Surgical Risk Calculator between the predicted risk and actual outcomes on average, and across the spectrum of predicted risk--from low to high. The researchers also evaluated performance on data subsets composed of higher-risk operations.

The results showed that the non-recalibrated Surgical Risk Calculator performed well, but there was a slight tendency for predicted risk to be overestimated for lowest- and highest-risk patients, and underestimated for moderate-risk patients; after recalibration, the distortion was eliminated. Calibration was also excellent for higher-risk operations, though observed calibration was reduced due to instability associated with smaller sample sizes. The study was published in the June 2016 issue of the Journal of the American College of Surgeons.

“Our study demonstrates that the NSQIP Surgical Risk Calculator has excellent calibration,” said study co-author Mark Cohen, PhD, statistical manager in the ACS Division of Research and Optimal Patient Care. “Although no such tool can provide perfect predictions, the Surgical Risk Calculator does what it is intended to do--accurately estimate a patient's probabilities for important adverse events postoperatively.”

Created in 2013, the NSQIP surgical risk calculator is a web-based decision aid and informed consent tool widely used by surgeons and their patients to calculate the risk of potential postoperative complications. The database includes comparative risk for more than 1,500 different surgical procedures. It currently receives an estimated 1,500 hits a day.

Related Links:
American College of Surgeons