EVALUATING THE IMPACT OF MACHINE LEARNING ON THE TIME NEEDED BY CLINICAL NURSE REVIEWERS TO IDENTIFY POSTOPERATIVE COMPLICATIONS

Teresa Hochstrasser*¹, Gerhard Halmerbauer¹, Marina Strobl², Stephan Winkler²
1 Process Management and Business Intelligence, University of Applied Sciences Upper Austria, Steyr,
2 Medical and Bioinformatics, University of Applied Sciences Upper Austria, Hagenberg, Austria

Introduction: To maintain the quality of surgical care, it seems essential to measure and compare the quality of health care
delivery in various institutions. The monitoring of postoperative complications after surgical interventions is of particular
importance here.
However, most expensive is the cataloging of risk factors and complications from Electronic Health Records (EHRs) by
clinical nurse reviewers. This process is time consuming and costly.
We developed a model to identify surgical patients with one or more postoperative complications using structured
patients’ EHR data and machine learning (ML) that correctly classified 76% of patients with a postoperative complication.
The aim of this study was to investigate, whether by using ML models to predict complications with a high probability
beforehand, the cataloging of complications might be facilitated and the time required for clinical nurse reviewers to
analyze a case be reduced.

Methods: In a randomized, controlled study, the influence of individual complication predictions per patient before the start
of the analysis on the respective time investment of our clinical nurse reviewers was examined.
A total of 1390 patients included in our study from January to June 2022 were randomly assigned to either the intervention
(n=682) or the control group (n=708). A prediction was generated for each patient in the intervention group as to whether
the patient had one or more complications. A list of predictions (complicative case yes/no) was provided for each clinical
nurse reviewer at the beginning of every month.
The clinical nurse reviewers measured the time needed for analyzing each case using a stopwatch in both groups.
Potential risk factors (type of operation performed, individual study nurse, ASA classification, complication y/n and number
of complications) for the time needed to review a case were analyzed using regression analysis in a subgroup of patients
without complications.

Results: The mean time (min - max) needed for the analyses of a case in the intervention group was 15.54 minutes (1.00-
117.00) vs. 15.69 minutes (4.00-97.00) for the control group (NS).
In a subgroup of patients without complications (n= 1092) the mean time (min - max) needed for the analyses of a case in
the intervention group was 12.27 minutes (1.00-52.00) vs. 12.60 minutes (4.00-59.00) for the control group (NS).
In the patients of the intervention group without complicative cases that were predicted rightly to be not complicative (=
true negative; n= 424) 10.85 minutes (1.00-30.00) was needed for analyses (p>0.001) vs. patients with an incorrect
forecast (= false positive; n= 105) 18.05 minutes (7.00-52.00) were needed for analyses (p<0.001).
Hernia repair (-2.74; p<0.001), colon surgery (6.92; p<0.001), rectum surgery (15.32; p<0.001) and prostatectomy (2.09;
p<0.001) vs. thyroidectomy, clinical nurse reviewer 1 (-1.91; p<0.001), prediction of complication false positive (1.82;
p<0.001) and true negative (-0.89; p<0.001) vs. control group were significantly associated with the time needed to review
a case.