Research On Visual Detection And Tracking Technology Of Surgical Instruments Based On Deep Learning

Visual detection and tracking of surgical instruments is one of the core technologies of surgical robots,which can assist clinicians or surgical robots to complete clinical operations.As early as the 1980 s,hardware-based detection and tracking of surgical instruments have been applied to clinical and surgical robots.The hardware-based method is relatively simple,but expensive hardware equipment and additional measures such as marking and aseptic processing make the actual operation more cumbersome.Therefore,vision-based surgical instrument detection and tracking algorithms are gradually being paid more attention,which are mainly divided into feature extraction algorithms and deep learning algorithms.Simple feature extraction algorithms cannot meet the requirements of accuracy and robustness.Deep learning methods are easily affected by interference factors such as blur,occlusion,and illumination during surgery,but they are still superior to traditional methods such as feature extraction in performance and effect,so they are the main research direction now.Various deep learning algorithms still have a lot of room for improvement in real-time,accuracy and robustness for the detection and tracking of surgical instruments.Therefore,based on practical application topics,the task of surgical instrument detection and tracking is focused in this paper,and a real-time and accurate algorithm based on deep learning is proposed.The main research contents of the paper are as follows:1.By consulting the literature,the theoretical basis and related algorithms of surgical instrument visual detection and tracking technology in recent years are comprehensively reviewed,and the performance differences of various deep learning algorithms are compared and analyzed.The analysis concludes that the technology will have great development prospects in terms of accuracy and real-time improvement in the future.2.The basic theories and project difficulties of surgical instrument detection and tracking are analyzed.The structure and basic principle of convolutional neural network and its training strategy are described,and the basics of object detection and tracking are discussed,including the types and processes of detection and tracking.In addition,the public data set used in this topic is analyzed in detail.3.A deep learning algorithm SI-YOLO suitable for surgical instrument inspection is proposed.The Sim AM attention mechanism is added to the backbone network of the original object detection YOLOv5 network model to improve the model’s feature extraction ability for surgical instruments in interference environments.In the neck network Neck,the Bi FPN node is used to replace the original PANet to improve the feature fusion strategy to solve the problem of inconsistent features of different surgical instruments.Finally,the latest α-Io U Loss is used to replace the original loss function to improve the speed and accuracy of the detection frame regression.The proposed SI-YOLO algorithm is trained and tested on the public data set m2cai16-tool-locations,and compared with the classic surgical instrument detection algorithm.The results show that the SI-YOLO algorithm proposed in this topic has the best performance.4.A surgical instrument tracking algorithm based on SI-YOLO combined with Deep Sort is proposed.Aiming at the problem of the object ID change in the scene of equipment disappearance and occlusion,on the basis of the original appearance feature extraction network Cosine of the Deep Sort tracking algorithm,the Inception-Res Net-v2 structure is introduced to replace the original Res Net residual network.Different feature map scales contain richer semantic information.The experimental results show that the surgical instrument tracking algorithm proposed in this subject has significantly improved tracking accuracy and precision,and the object ID transformation frequency when the instrument disappears or is occluded is significantly reduced.5.In order to realize the preliminary application of the surgical instrument detection and tracking algorithm based on deep learning,we built a simulated surgical experiment platform and created a data set.The trained model was deployed in a real experimental environment for real-time detection and tracking,and good results were achieved.