Virtual Reality Surgery Tracking for Teleoperated Surgeries

 Team: Yiren Wang, Zi Ye
Mentor: Blake Hannaford (ECE)

Improving the safety of high-risk surgeries has always been a worldwide problem. Nowadays, there are many well-established medical methods focusing on this issue such as computed tomography (CT) scans, endoscopy, and surgical tracking systems. However, all of these systems share a common limitation: they use preoperative images or 2D videos which fail to provide comprehensive and real-time information for surgeons remotely controlling robotic surgical devices. With our novel system, we aim to improve the limitation of existing surgical tracking systems with AR and VR tracking methods.

Our goal was to develop a VR system in a surgical environment. The idea of a VR tracking system is to abandon current technology: instead of using a camera to record real-time videos, VR tracking systems would allow surgeons to perform teleoperated surgeries within a specified 3D environment. By wearing the VR headset, surgeons can see the exact position of the simulated medical instruments and investigate any internal organs or large area of trauma on patients in real-time without any inconveniences or visual limitation. The VR surgical tracking system not only has higher reproduction, clearer resolution and easier manipulation, but it also makes high accuracy and low-cost room-scale 3D tracking possible.