Virtual Reality Surgery Tracking for Teleoperated Surgeries


Yiren Wang
Zi Ye

Improving the safety of high-risk surgeries has always been a worldwide problem. Nowadays, there are many well-established medical methods focusing on this issue such as computed tomography (CT) scans, endoscopy, and surgical tracking systems. However, all of these systems share a common limitation: they use preoperative images or 2D videos which fail to provide comprehensive and real-time information for surgeons remotely controlling robotic surgical devices. With our novel system, we aim to improve the limitation of existing surgical tracking systems with AR and VR tracking methods.

Our overall goal is to develop a VR system in a surgical environment. The idea of a VR tracking system is to abandon current technology: instead of using a camera to record real-time videos, VR tracking systems would allow surgeons to perform teleoperated surgeries within a specified 3D environment. By wearing the VR headset, surgeons can see the exact position of the simulated medical instruments and investigate any internal organs or large area of trauma on patients in real-time without any inconveniences or visual limitation. The VR surgical tracking system not only has higher reproduction, clearer resolution and easier manipulation, but it also makes high accuracy and low-cost room-scale 3D tracking possible.

We are currently in a process of verifying the precision and accuracy of HTC Vive system by comparing with two other well established surgical tracking systems, Medtronic and Stryker. We concluded that HTC Vive system has a similar 3D positional accuracy and real-time tracking precision as Medtronic and Stryker system.

Mentored by:

Blake Hannaford