DIAGNOSTIC ACCURACY OF AN AI-DRIVEN PRIORITISATION TOOL FOR THE IDENTIFICATION OF HIGH-CONFIDENCE NORMAL CT HEAD STUDIES

 

Jack Sheppard 1, Stefan T. Schwarz 1,2, Hisham Jaber 1,3, Christopher Goodwin 1,3, Mel Ryan 4,Alicja Raginis-Zborowska 4, Thomas Pickersgill 4, Caroline Parker 1, Dane Evans 1, Phillip Wardle1,3, Kieran G. Foley 1,3

 

1 National Imaging Academy Wales (NIAW)  2 Cardiff & Vale University Health Board   3 Cwm Taf Morgannwg University Health Board  4 Harrison.ai

 

Purpose

Radiology departments are struggling to meet the increasing demand for emergency CT head imaging. Many rely on costly teleradiology companies to provide timely reports. We evaluated the utility of an AI-prioritisation tool to identify high-confidence clinically normal non-contrast CT head studies. The primary aim was to test the agreement between the AI-prioritisation tool and reference index for categorisation. The secondary aim was to determine the standalone diagnostic accuracy (sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV)) of the AI-prioritisation tool for categorisation of “Urgent” studies.

 

Methods and Materials

Consecutive CT studies performed out-of-hours in two University Health Boards (UHBs) in July 2025 were included in this retrospective study. Images and reports were anonymised and transferred to a secure PACS where an AI tool inferenced all studies. The AI-prioritisation tool categorised each CT as “Urgent” or “Not Urgent” based on the clinical acuity of findings. A consultant neuroradiologist reviewed each CT providing the reference index classification (“Urgent” or “Not Urgent”). All studies were originally reported by either a resident radiologist or teleradiologist. The consultant neuroradiologist assigned their classification before reviewing the original report. “Urgent” reports were defined as any CT with positive findings that may have influenced clinical management out-of-hours. The primary outcome was the agreement between the AI-prioritisation tool and consultant neuroradiologist, defined as the proportion of cases correctly categorised. The study abstract was written in accordance with the STARD abstract guidelines for diagnostic accuracy studies.

 

Results

 

615 CT head examinations were included (UHB1=443, UHB2=172). 166/615 (27.0%) were prioritised as “Urgent” by the AI-prioritisation tool. 94/615 (15.2%) were prioritised as “Urgent” by the consultant neuroradiologist. Agreement was 86% (95% CI [83, 88%]). Sensitivity, specificity, PPV, and NPV of the standalone diagnostic tool for classification of “Urgent” studies were 91% [84-96%], 85% [81-88%], 52% [44-60%], and 98% [97-99%], respectively. Correct categorisation of “Urgent” by the AI-prioritisation tool, but for the incorrect reason, was demonstrated in two cases (0.3%) (Figure 2).

 

Conclusion

The AI-prioritisation tool demonstrated good agreement, sensitivity and NPV compared to consultant neuroradiologists. Further prospective validation is required, however, the AI-prioritisation tool showed the potential for safe identification of high-confidence normal CT head studies. Of the abnormalities in the false negative group, cervical spine fractures made up the greatest proportion. The AI-prioritisation tool may enable deferred reporting of cases prioritised as “Not Urgent”, potentially reducing costs associated with outsourcing, and acknowledging specific limitations.