Soft Tissue Motion 3D Tracking Based On Deep Feature Learning

Modern medicine often uses endoscopes to examine,treat and monitor the human body.However,due to the narrow field of view of the endoscope and the deformation of the human body due to breathing,movement,etc.,it is difficult for surgeons to fully and accurately restore and track the surface of the soft tissue during the operation.At the same time,it is difficult to track the surface feature of the soft tissue in the endoscopic image sequence.,It also has important applications in the fields of surgical training and teaching,virtual reality soft tissue 3D modeling.In this paper,the method of soft tissue 3D motion tracking based on deep feature learning is researched,which mainly includes three parts:inter-frame triangle matching,deep feature extraction and matching,and intra-frame tracking.The details are as follows:(1)A method of soft tissue surface feature tracking based on triangle matching is proposed.This method starts with the description of the feature.The feature is the gray-scale collection area on the surface of the soft tissue—blob,which includes the newly proposed triangle search division method,and innovatively uses the center of gravity coordinate theory to quickly and accurately obtain the blob matching data between frames.The purpose of tracking the surface features of soft tissues without marking and few samples is achieved.(2)A method of tracking soft tissue surface features based on deep matching network is proposed.This method expands on the basis of the triangle matching algorithm.First,the first N frames of inter-frame blob matching data obtained by the above triangle matching algorithm are used to construct the self-made sample set required for training the deep matching network.The deep matching network is pre-trained on the OCL face data set,and then trained on the self-made training set.After the training is completed,the blob matching is performed in the subsequent frames,and the blob matching matrix between the subsequent frame and the first frame is obtained.The result of inter-frame feature matching can be obtained,that is,the inter-frame blob tracking is completed.And on this basis,the results of this method are compared with the matching results based on convolutional neural networks.The experimental results show that the method proposed in this paper has a higher matching accuracy.(3)Finally,the triangle matching algorithm is used to match the left and right views in the frame,and the geometric relations and parameters under the binocular endoscope are used on the matching results to restore the three-dimensional space coordinates of the blobs,and calculate the coordinates of the same blob in different frames to achieve soft tissue Three-dimensional motion tracking of the surface.Experiments were carried out on the video sequence of the endoscope,and the experimental results showed that the blob tracking results were consistent with the movement of the soft tissue surface.