Title: From the Eye of The Learner: Assessing Learning and Proficiency with Eye-Tracking Metrics During Simulated Ultrasound-Guided Regional Anesthesia.

Authors: Arun Muthukumar M.D.¹; Juan Jose Guerra Londono M.D.¹; Audrey Tacderas B.S.¹; John D. Mitchell M.D.^1,2; Adnan Hussain, M.D^1,2; Carlos Eduardo Guerra Londono M.D.^1,2.

Affiliation:

1-    Department of Anesthesiology, Henry Ford Hospital, Detroit, Michigan, USA – 48202.

2-    Michigan State University CHM, Grand Rapids, Michigan, USA - 49503

 

Abstract:

Background:
Ultrasound-guided regional anesthesia (UGRA) requires precise hand-eye coordination during real-time tasks [1]. Skill acquisition in UGRA remains poorly understood, partly due to the absence of standardized objective metrics. Recently, eye-tracking emerged as a potentially useful tool for skill acquisition in medicine. In anesthesia, studies of eye-tracking metrics are limited to classic epidural anesthesia and UGRA image interpretation [1, 2]. This study aimed to objectively describe eye-tracking metrics among anesthesiology residents undergoing simulated UGRA procedures before, during, and after a regional anesthesia rotation. We hypothesized that eye-tracking metrics could differentiate proficiency and identify skill acquisition during the rotation.

Methods:
The present study was approved by the IRB at Henry Ford Health. As part of a comprehensive curriculum on UGRA, anesthesiology residents were recruited to perform simulated UGRA procedures for interscalene, femoral, and sciatic popliteal peripheral nerve blocks at three distinct sessions: before, during, and after their scheduled rotation month. Participants gave informed consent for eye-tracking recordings during the course. Tobii 3 Glasses (Tobii Technologies, Ltd., Stockholm, Sweden) were employed to record gaze patterns, focusing on gaze stability (measured by saccade frequency as an indicator of erratic eye movement) and fixation metrics under identical conditions using protocolized start and stop points and a standard sequence of probe fixation, needle insertion, and nerve localization. Trainees were required to verbalize recognition of anatomical structures and the final position of the needle’s tip. An investigator reviewed each trial (3 for each block/session) and scored performance using a global rating scale, as described by Wong et al. Recordings not following protocol and those with suspected miscalibration were excluded.

Discussion:

This study demonstrated that eye-tracking metrics can serve as an objective measure of learning and proficiency in ultrasound-guided regional anesthesia (UGRA) among anesthesiology residents. Eye-tracking has proven valuable in assessing workload, visual attention, and situational awareness during regional anesthesia. Metrics - pupil size reflect changes in workload and stress levels, while fixation duration indicates cognitive load.

In UGRA, specific metrics including total gaze time in and outside the area of interest (AOI), and time to first fixation on the AOI are used to differentiate between novices and experts. Experts typically demonstrate shorter total times and faster times to first fixation on the AOI, with reduced gaze time outside the AOI compared to novices. Gaze shifts and gaze ratio further illustrate the prioritization of visual attention, with experienced anesthetists exhibiting more efficient monitoring behaviors.

These objective insights into cognitive processes not only enhance training but also contribute to improving patient safety by promoting better procedural performance. [1] Eye-tracking technology has the potential to refine UGRA training programs, providing targeted feedback and facilitating faster skill acquisition. As medical education evolves, incorporating such advanced tools can further bridge the gap between simulation and real-world practice, ultimately benefiting both trainees and patients.

 

Results:
A total of 174 recordings were analyzed, of which only 102 met the criteria for analysis. These recordings included a total of 12 anesthesia residents: (1) before the rotation, referred to as pre/out of rotation (n=25), (2) midway through the rotation, referred to as mid (n=34), and (3) after completing the rotation, referred to as post (n=43). Our analysis revealed significant differences in average fixation duration and saccade patterns as residents progressed through the rotation. Specifically, the average fixation durations increased from 1074 milliseconds pre-rotation to 1079 milliseconds mid-rotation, and 1117 milliseconds post-rotation. (Figure 1A and B) Additionally, saccade counts decreased from 30.29 at the pre-rotation stage to 30.12 mid-rotation and further to 28.45 post-rotation, indicating more focused and efficient gaze behavior (Figure 2). Furthermore, the number of saccades, total duration of fixations, and average duration of whole fixation showed significant differences among the 5 different levels of performance.

 

Conclusion:

In a cohort of anesthesia residents performing simulated UGRA procedures, eye-tracking metrics were able to visually differentiate different levels of UGRA proficiency. At the pre-rotation stage, residents showed frequent misalignments, higher saccade counts, and shorter average fixation durations on relevant areas, leading to less accurate procedural execution.
Post-rotation, residents exhibited enhanced gaze fixation. This study demonstrates the potential of head-mounted Tobii 3 Glasses to provide quantifiable insights into eye-tracking metrics as a measure of skill development and proficiency in UGRA.

 

References:

1. Borg LK, Harrison TK, Kou A, Mariano ER, Udani AD, Kim TE, Shum C, Howard SK; ADAPT (Anesthesiology-Directed Advanced Procedural Training) Research Group. Preliminary Experience Using Eye-Tracking Technology to Differentiate Novice and Expert Image Interpretation for Ultrasound-Guided Regional Anesthesia. J Ultrasound Med. 2018 Feb;37(2):329-336. doi: 10.1002/jum.14334. Epub 2017 Aug 4. PMID: 28777464.
2. Novice and Expert Anesthesiologists' Eye-Tracking Metrics During Simulated Epidural Block: A Preliminary, Brief Observational Report. Local Reg Anesth. 2020 Aug 21;13:105-109. doi: 10.2147/LRA.S267879. PMID: 32922069; PMCID: PMC7457852
3. Wong DM, Watson MJ, Kluger R, Chuan A, Herrick MD, Ng I, Castanelli DJ, Lin LC, Lansdown A, Barrington MJ. Evaluation of a task-specific checklist and global rating scale for ultrasound-guided regional anesthesia. Reg Anesth Pain Med. 2014 Sep-Oct;39(5):399-408. doi: 10.1097/AAP.0000000000000126. PMID: 25140509.