Performance of AI-Assisted Diagnostic Systems Versus Radiologists in Prospective Breast Cancer Screening

Nahid. Nafisi, Mandana Pourian, Elham. Keshawarz, Mehran. Arab Ahmadi , Rassa. Ghavami Modegh, Hamed. Dashti, Masoumeh Gity. Presenter: Elham Keshawarz

• To prospectively assess and compare the diagnostic accuracy of two commercial artificial intelligence (AI) tools, CommaMed and AIMEDIC, with conventional radiologic interpretation for breast cancer screening patients. • A prospective study was done with 171 patients undergoing screening mammography. The performances of human radiologists were compared with those of two different AI models (CommaMed and AIMEDIC). Main diagnostic measures were derived: Area Under Curve (Receiver Operating Characteristic), Sensitivity, Specificity, Accuracy, False Positive Rate (FPR), False Negative Rate (FNR), and Recall Rate. The confusion matrix was derived for all models to describe classification performances. • The AUC achieved by radiologists was 0.821 with perfect sensitivity (1.000), specificity of 0.642, and recall rate of 38%. The CommaMed AI system performed better than the other two systems with an AUC value of 0.879, specificity of 0.758, accuracy of 0.766, and lower recall rate of 27%, while sustaining perfect sensitivity. On the other hand, the discriminative power of the AIMEDIC system was lower with an AUC value of 0.633, and high false positives leading to a large recall rate of 74%. The FNR of all models was 0.0, signifying • There is considerable variation among AI systems' performances with regard to breast cancer screening. The CommaMed system has achieved optimal trade-offs between high sensitivity and enhanced specificity to suggest its utility as an additional valuable tool for radiologists. Its application could increase efficiency and minimize unnecessary follow-up screenings with regard to breast cancer screening programs. • The incorporation of efficient AI models such as CommaMed into the breast cancer screening process can help radiologists by increasing specificity and decreasing recall rates to ultimately increase efficiency and reduce costs and anxiety related to false positives. no false negatives or missed malignant cases