Research On Binocular Endoscopic Image Reconstruction Based On Deep Learning

At present,there are still many challenges to obtain the depth information of the lesion area through endoscopic images in the process of minimally invasive surgery.The characteristic of the two-dimensional image obtained by endoscope makes it difficult for the surgical robot to accurately recover the three-dimensional spatial structure of the lesion,which further restricts the development of the intelligent surgical robot.To solve this problem,this paper carried out the research on binocular endoscopic image reconstruction technology based on deep learning.The main contents are as follows:1.Aiming at the problem that the disparity estimation network based on deep learning has low reconstruction accuracy on endoscopic images,this paper designs a Encoder-decoder to solve the problems that endoscopic images have little texture and are easy to be occluded.Firstly,to increase the receptive of the model,we use several dilated convolution layers,which aims to increase the perception ability.Secondly,we use a multiple hierarchy architecture to extract the features in different scales.And use the fused features for decoder,which taking the context information into account.And the last part is in the loss function,this paper adds the consistency constraints of left and right disparities to loss function.This item can use the constraint information of stereo fully.2.In the view of the dynamic characteristics of endoscopic image,this paper proposes a frame by frame of optimization method to complete the disparity estimation task.In this method,we firstly train out an experienced deformation model in a form of single-layer perceptron,and then use this model to disparity estimation in each frame.In order to improve the accuracy of the deformable model,we apply it on each frame in an iterate way.And the experical deformation model derives from a traditional spline interpolation model named Thin-plate Spline interpolation(TPS).Meanwhile,it was expressed in the form of a single-layer perceptron,use which can train out an experical deformation model on the special region of edoscopic images.The experical deformation model is much more explicit because it was trained on the endoscopical data.In addition,for retaining the smooth deformation characteristic of TPS when training the model,this paper proposes to use an alternate training way to train model parameters.The experimental results show that the empirical deformation model trained by this method not only retains the smoothness of the TPS model,but also improves the ability of disparity estimation for the details of the region of interest.