119 posters, 6 topics, 524 authors, 243 institutions

RCR Global AI Conference 2026

29-30 June, 2026 | QEII Centre, Westminster

134

Mind the Gap: Assessing the Generalizability of a Public AI Model for Intracranial Hemorrhage Detection on Non-Contrast CT

View Poster Contact Presenter

Poster Presenter

Ali Mansoor

Authors

Rabia Shaukat, Ali Mansoor, Fatima Iqbal, Muhammad Arslan, Muneeb Ahmad, Nighat Haroon Khan

Affiliations

Lahore General Hospital, Eastern Medical Technology Services, Lahore General Hospital / Ameer ud Din Medical College/ Post Graduate Medical Institute, Lahore General Hospital / Ameer ud Din Medical College / Post Graduate Medical Institute, Lahore General Hospital/ Ameer ud Din Medical College / Post Graduate Medical Institute, Jinnah Hospital Lahore

Part of Topic

Transforming practice and leadership: pilot or test data on implementation of AI into clinical practice, clinical feedback or patient perspectives

Background:

Spontaneous intracranial haemorrhage (ICH) carries high mortality (30–50%) and significant long-term disability (1,2). In low- and middle-income countries such as Pakistan, delayed diagnosis is worsened by limited access to expert imaging interpretation (3). Early detection on non-contrast CT is therefore critical. Artificial intelligence (AI), particularly deep learning models, has shown strong performance in ICH detection, with reported AUCs of 0.85–0.96 (4). However, approved AI tools remain limited in LMIC settings, and real-world comparative data against clinical readers in Pakistan are scarce.

Objective:

To compare the diagnostic accuracy of an AI algorithm with radiologists of varying experience levels in detecting acute ICH on non-contrast CT scans.

Materials and Methods:

Following Ethical Review Board approval, this pilot retrospective study included 100 non-contrast CT brain scans (47 ICH-positive, 53 negative) from a tertiary trauma referral centre in Lahore, Pakistan. A radiology resident, junior consultant, and senior consultant independently reviewed anonymized scans for ICH presence and subtype, blinded to reports and each other’s assessments. Ground truth was established by consensus of two senior consultant radiologists.For AI analysis, anonymized DICOM images were processed using a deep learning pipeline in Google Colab based on a 2D EfficientNet-B3 convolutional neural network, pretrained on ImageNet and adapted from the RSNA Intracranial Hemorrhage Detection Challenge. Diagnostic performance was assessed using sensitivity, specificity, PPV, NPV, and inter-observer agreement, with AI evaluated at the default 0.5 threshold and optimized cut-offs using the Youden Index and a maximal-sensitivity approach.

Key Results:

•At default threshold of 0.5, AI 57% sensitivity, 43% specificity, and 50% accuracy.

•Youden-optimized cut-off of 0.46 – sensitivity ↑ to 59.6% and specificity ↑ to 54.7%.

•Maximal-sensitivity strategy -- sensitivity ↑ to 83% but specificity ↓ to 17%.

•McNemar tests confirmed that all human readers significantly outperformed AI (p < 0.001)

•Comparisons among human readers revealed no significant differences (p > 0.05)

•AI had:

Highest false-negative rate (71.4%) for epidural hemorrhage

Intraparenchymal hemorrhage - most misses numerically (21)

Overcalls most common for intraventricular hemorrhage (26.4% )

Discussion:

•Literature on AI performance in ICH detection is mixed : some studies report high diagnostic performance (5) with a comparative study concluding that AI outperforms radiologists (6) while others have found that radiologists are still better - including a prospective multicenter study (7); others report that residents outperform AI (8) even under time pressure (9)

•Consistent with our findings of low specificity of AI, even studies that report high sensitivity report lack of specificity in comparison to human readers (10)

•A meta-analyses identified epidural hemorrhage as particularly challenging to detect with AI, consistent with our highest false-negative rate (71.4%) in this category (11).

•Prior studies also demonstrate reduced detection of subdural and subarachnoid hemorrhages, supporting the variability in AI performance seen in our error patterns (12).

•Reported improvements in AI performance with larger hemorrhage size and higher acuity (13) and multiplicity (8) may explain missed subtle or small lesions in our cohort.

•The lower sensitivity of AI for ICH detection in our study, highlights the limitations of current algorithms as standalone diagnostic tools. Similar concerns in the literature emphasise the need for robust external validation and real-world clinical trials before routine implementation (14) and about generalizability of these tools with indeterminate failure modes (15).

Conclusion:

This study demonstrates a clear generalizability gap in AI performance on local CT brain imaging data, with reduced accuracy compared to expected performance in published settings. Improved sensitivity after threshold adjustment was offset by poor specificity. Local validation and adaptation are essential before clinical use.