Improvement of the correctrate of detection of surgical comorbidities

INTRODUCTION：

In response to the active clinical development of AI to assist in clinical practice, hospital accreditation requires hospitals to have a discussion of comorbidities and literature on the possibility of comorbidities in any disposition, serious comorbidities may jeopardize the safety of the patient and cause a waste of healthcare resources, and some of the comorbidities can be prevented, and prevention in advance can enhance the prognosis of treatment.

However, there is a lack of a prompt assistive mechanism in the care process to provide real-time gatekeeping. With the consensus of the system and the indication of the supervisor, a cross-disciplinary team will be assembled to intervene and improve the situation, hoping to enhance the use of AI in surgical comorbidities and to innovate assistive healthcare.

METHOD：

1.Audit results showed that the current back-end comorbidity notification system had incorrect and missing case accuracy: only 48.8% of surgical comorbidity cases were correctly matched (21/43) and 0% of surgical comorbidity cases were checked in the discharge summary for unplanned surgical reentry cases (0/43).

2.Considering that the AI-assisted target group is the physician community and analysing the textual characteristics of the medical record detection, the key diagnostic groups of cases that meet the ‘standard medical record’ of surgical comorbidities through the classification of discharged medical records were selected for AI detection.

3.To assist clinicians in the care process through AI detection, the following countermeasures were implemented:

   Policy group 1: Establish the scope of detection: intervene in the list of new surgeons, define the scope by analysing the big data of four peer hospitals, and use the top five surgical comorbidities in four specialties and five surgical comorbidities as the cornerstone to establish a surgical comorbidity monitoring model.

  Policy group 2: Establishing a detection and verification model: physician-centred assistance in clinical care planning, team meetings to agree on implementation methods, telephone and real-time communication to resolve implementation concerns, and experimental methods to establish appropriate AI detection and implementation steps.

  Policy Group 3: Build AI reminder screen: including detailed download, real-time notification and reminder mechanism function building.

RESULTS：

1.2022.04 Phase I Critical Diagnostic AI Detection Validation Sensitivity ≥90% Pass Rate at 100% Target Value

2.2022.08 Phase III AI clinical testing critical diagnosis validation sensitivity ≥90% pass rate of 100% target value

3.2023.07 AI reminder screen countermeasure execution rate 100% (4/4), actual use in clinical.

(1)Comorbidities Key Diagnostic Categories Detection Pass Rate:2023.6.15 Third Key Diagnostic Group 95.0% (93/98) detection pass rate with a sensitivity of 90% or above.

(2)Information prompt screen standard, with the category ‘Infection’ sensitivity 90%, accuracy 8.5% first try, with 3 conditions for the parallel development of the benchmark, including at least two keywords appear at the same time, sensitivity (sen) > 0.9, Precision) > 0.9

CONCLUSIONS：

In this activity, the Surgical Comorbidity Detection Module will be built in 2022 for 5 comorbidity categories in 4 specialties, but there are still 7 specialties and 93 comorbidity categories that need to build the detection model based on the module and to carry out a levelling-out of the actual clinical use, in order to promote the alert of suspected or occurred comorbidity cases and the future development of the Surgical Comorbidity Prediction Module.