Atoum, J.; Park, J. K.; Akter, M.; Kavoussi, N.; Wisniewski, P.; & Wu, J. (2026). Focus on the experts: Co-designing an augmented reality eye-gaze tracking system with surgical trainees to improve endoscopic instruction. Human Factors in Healthcare, 9, 100115. https://doi.org/10.1016/j.hfh.2025.100115
Surgical training traditionally follows an apprenticeship model in which trainees learn by working directly with experienced surgeons in the operating room (OR). While this approach is effective, it requires constant supervision and does not scale well as the demand for trained surgeons continues to grow. For many endoscopic procedures, teaching happens during live surgeries, where patient care must take priority. This limits trainees’ opportunities to practice freely, make mistakes, and receive detailed feedback. Augmented reality (AR) has the potential to improve surgical training efficiency, but more research is needed to understand what trainees actually need from AR-based systems.
In this study, we worked with 18 surgical trainees to explore the strengths and limitations of their current training environment and to co-design an AR eye-gaze tracking system tailored to their needs. Trainees emphasized that one of the most difficult skills to master is translating two-dimensional endoscopic images into a three-dimensional understanding of patient anatomy. They felt that an AR system showing where an attending surgeon is looking could help them better understand anatomy and surgical decision-making in real time. The trainees viewed an AR-based eye-gaze tracking system as a valuable supplement to current training that could enhance learning during OR cases without interfering with patient care. By incorporating trainee feedback, this work provides user-informed design guidelines for future collaborative AR training tools in endoscopic surgery.
Fig. 1. (a) A screenshot of the video displayed to the participants, overlaying a yellow sphere on a pre-recorded video of a simulated stone identification task. (b) Illustration of how expert and surgical trainees would communicate through visual guidance in an AR-based eye-gaze tracking.