Eye tracking technology differentiates novice and expert image interpretation for ultrasound-guided regional anesthesia
Historically, educational research has relied on test-retest and self-assessment methods to infer learning. Similarly, training programs use subjective evaluations from instructors to determine trainees’ milestone achievement and competence. Objective measures of learning are needed to guide the individual learner’s pathway from novice to expert. Eye tracking has been used in surgery, radiology, nursing, and athletics for both training and assessment. Based on a recent feasibility study in the field of ultrasound-guided regional anesthesia (UGRA), we designed this study to test the hypothesis that eye tracking may differentiate novices from experts in image interpretation for UGRA.
This study was approved by the IRB and VA research committee, and participants provided written consent. We recruited a convenience sample of CA-1 residents (novice group) and UGRA faculty (expert group) from a single university-affiliated Veterans Affairs hospital. Recruits were excluded if they could not be successfully calibrated on the eye tracking glasses. We collected baseline survey data related to prior ultrasound experience.
Experimental Setting: Participants were seated in front of a 50-inch screen and fitted with eye tracking glasses (Tobii, Karlsrovägen, Sweden). Room lighting and set-up were the same for each participant (Figure 1).
Examination: After calibration, 5 UGRA sonograms were serially projected as a slideshow (Powerpoint, Microsoft Office, Redmond, WA). Images were selected by an investigator located at a different institution and uninvolved in data collection (ADU) from images included in a UGRA handbook. Participants were asked one standardized anatomy-based per image while their eye movements were recorded. Questions and answers were developed and iteratively tested by four investigators who were not study recruits. The answer to each question was a location on the image defined as the “area of interest” (AOI; Figure 2).
Outcomes: Total gaze time in the AOI (sec) was primary. Secondary outcomes were: total gaze time outside the AOI (sec), total time to answer questions (sec), and time to first fixation on the AOI (sec).
Analysis: Gaze data were analyzed using Tobii Pro Lab Analyzer (Karlsrovägen, Sweden). Statistical analyses were performed using NCSS-PASS software (Kaysville, UT).
12 participants, 6 novices and 6 experts, were recruited. One novice and one expert were excluded due to unsuccessful calibration of the eye tracking glasses; this was attributed to their prescription eye glasses. The primary outcome, total gaze time (mean±SD) in the AOI, was not different between groups (7±4 seconds for novices and 7±3 seconds for experts; p=0.150). Gaze time outside the AOI (unfocused time) was greater for novices (75±18 seconds) vs. 44±4 seconds for experts (p=0.005). Total time to first fixation in the AOI and total time to answer the five standardized questions and were both shorter for experts as compared to novices (Figure 3). Experts answered 3±1 questions correctly compared to 2±1 for novices (p=0.074).
Qualitatively, expert gaze was more focused. In the supraclavicular image (Figure 2), participants were asked where they would place local anesthetic for a 5th digit fracture open reduction and internal fixation. While both groups spent time dwelling within the AOI, expert gaze tracked more quickly to the subclavian artery, brachial plexus, and “corner pocket” (Figure 2). In comparison, novice gaze generally spent more unfocused time fixating on irrelevant surrounding structures
Experts in UGRA take less time to identify sonoanatomy of interest and exhibit a more focused gaze pattern compared to novices. Eye tracking is a potentially useful tool to differentiate novices from experts in the domain of ultrasound image interpretation, a key skill in field of UGRA, and may represent an objective measure to benchmark a trainee’s progress towards expertise.