Feature Based 3D Visual Reconstruction And Tracking Of Soft Tissue

Endoscopy is a part of routine clinical practice for minimally invasive examination and treatment.Compared with traditional thoracotomy,minimally invasive surgery has the characteristics of minimal trauma,short hospital stay,less intraoperative pain and rapid postoperative recovery.However,due to the narrow scope of endoscopic observation,it is very difficult for surgeons to determine the position of endoscope and perceive the surgical instruments during surgery.Three dimensional reconstruction technology can be used to restore the three dimensional shape of the surgical area,which is of great help to the analysis during and after surgery.In this paper,a feature-based soft tissue three dimensional reconstruction and tracking method is studied,including feature detection,feature matching,three dimensional surface reconstruction and feature tracking.First,feature detection is carried out on the endoscopic images.In this paper,feature detection is divided into feature extraction and feature description.In the feature extraction part,the FAST feature extraction algorithm is studied in detail.In view of the poor classification effect of the constructed decision tree,the decision tree was constructed based on C4.5 algorithm.At the same time,the original data was divided for the construction of double decision tree,which made the feature extraction performance more stable and feature point extraction more efficient.In the feature description part,the FREAK descriptor is studied,which is combined with the improved feature extraction algorithm in this paper to extract feature points in the scale space and conduct quadratic function fitting according to the response scores of feature points in different scales to obtain the scale invariant descriptor of sub-pixel accuracy.The experimental results on endoscopic images show that the feature extraction method in this paper has higher extraction quantity and faster extraction speed,and the feature description algorithm in this paper has higher computational efficiency.Secondly,this paper proposes a Convolutional Neural Network(CNN)based feature matching method.Aiming at the problem of poor robustness of traditional feature matching methods in endoscopic images,a feature matching method based on CNN is designed.The training data set of feature points is generated on the initial frame of a certain length in the endoscope video,and the data set is used for CNNtraining.The corresponding classification results of feature points in the subsequent matched frames are obtained through the trained CNN,and then feature matching is realized according to the classification results of feature points in the matched frames.Through experiments on endoscopic images,the proposed method has higher accuracy than other matching methods and well directional invariance and scale invariance.Finally,binocular geometric constraints and Delaunay triangulation are used to reconstruct the three-dimensional surface and track the feature points.The relations between the two frames of feature point matching and parallel type binocular camera parameters are used to restore the three dimensional coordinate of feature points.The spatial discrete points are projected on a two dimensional plane and the point-by-point interpolation Delaunay triangulation is used to get the surface triangulation.The three dimensional surface is then reconstructed combing with texture mapping.Feature tracking is realized through calculating the same feature point position in different frames.The experiment was performed on endoscope video,the reconstruction results were consistent with the soft tissue surface shape and the feature tracking results were consistent with the soft tissue movement.