A Fast, Low-Cost, Computer Vision-Based Approach For Tracking Surgical Tools

The number of Robot-Assisted Minimally Invasive Surgery (RMIS) procedures has grown immensely in recent years. Like Minimally Invasive Surgeries (MIS), RMIS procedures provide improved patient recovery time and reduced trauma due to smaller incisions relative to traditional open procedures. Given the rise in RMIS procedures, several organizations and companies have made efforts to develop training tasks and certification criteria for the da Vinci robot. Each training task is evaluated with various quantitative criteria such as completion time, total tool path length and economy of motion, which is a measurement of deviation from an `ideal' path. All of these metrics can benefit greatly from an accurate, inexpensive and modular tool tracking system that requires no modification to the existing robot. While the da Vinci uses joint kinematics to calculate the tool tip position and movement internally, this data is not openly available to users. Even if this data was open to researchers, the accuracy of kinematic calculations of end effector position suffers from compliance in the joints and links of the robot as well as finite uncertainties in the sensors. In order to and an accurate, available and low-cost alternative to tool tip localization, we have developed a computer vision based design for surgical tool tracking. Vision systems have the added benefit of being relatively low cost with typical high resolution webcams costing around 50 dollars. We employ a joint geometric constraint - Hough transform method for locating the tool shaft and subsequently the tool tip. The tool tracking algorithm presented was evaluated on both an experimental webcam setup as well as a da Vinci Endoscope used in real surgeries. This system can accurately locate the tip of a robotic surgical tool in real time with no augmentation of the tool. The proposed algorithm was evaluated in terms of speed and accuracy. This method achieves an average 3D positional tracking accuracy of 3.05 mm and at 25.86 frames per second for the experimental webcam setup. For the da Vinci endoscope setup, this solution achieves a frame rate of 26.99 FPS with an average tracking accuracy of 8.68 mm in 3D and 11.88 mm in 2D. The system demonstrated successful tracking of RMIS tools from captured video of a real patient case.