Develop an Automated Deep Learning Method of AOI Identification in Surgical Videography (DSI-SRP)
This DSI-SRP fellowship funded Tingyan (Nicholas) Deng to work in the laboratory of Professor Benoit Dawant in the Department of Electrical Engineering and Computer Science during the summer of 2020. Tingyan is majoring in Computer Science, Math, and Economics, and is minoring in Business and Scientific Computing.
The project funded by this fellowship aimed to discover new solutions for automating the capture of high-resolution videos for surgical procedures. There is increasing professional, societal, and legislative interest in video records in surgery, yet there is no existing solution for automating the capture of high-resolution videos for the majority of surgical procedures. As part of an ongoing VISE-affiliated collaboration between VISE faculty and a VUMC surgeon, the Surgical Analytics Lab has prototyped a novel surgeon-worn video platform that provides an unobstructed view of the surgical field; a key consideration now is to develop methods to identify where the surgical “action” (area of interest; AOI) is taking place in the frame of these videos. This is important for developing tracking mechanisms that can automatically adjust the camera to maintain vigilance in the surgical field despite the natural body movement of the wearer. The goal of the project is to use deep learning to develop methods for automatic AOI detection systems.
Throughout the fellowship and last year, Tingyan developed a deep learning pipeline that not only tracts the surgical wound, but also a surgical instrument, (i.e., bovie) and has written two conference papers on the two individually. This project has grown and moved forward through the past year and we are expanding our system to a greater extent. Tingyan continues to try new machine learning algorithms and methods to improve his previous model while finding new tools to make our future model more comprehensive (i.e. more features) and efficient. We hope this project can be implemented in actual surgeries in the future.
Earlier this year, Deng’s research was featured in Research News @ Vanderbilt. https://news.vanderbilt.edu/2021/04/09/student-developed-machine-learning-techniques-make-surgeries-safer-and-easier-to-review/
In addition to receiving support through a DSI-SRP fellowship, this project was supported and facilitated by the DSI Data Science Team through their regular summer workshops and demo sessions.