Research On Depth Estimation Method Based On Liver Monocular Endoscope

Monocular endoscopic minimally invasive surgery is currently the most commonly used method to replace traditional open surgery.For example,laparoscopy and other therapeutic equipment are introduced into the human abdomen to diagnose and treat the liver.During the operation,the twodimensional images presented by the laparoscope cannot provide the doctor with accurate position information.The depth estimation can provide accurate distance information for the laparoscopic three-dimensional surgery navigation system to help the doctor perform the operation accurately.However,the limited field of vision in the human body,the sparse texture of organs and tissues,and the high reflectivity of the tissue surface bring certain difficulties to the depth estimation of the endoscope.In recent years,depth estimation in natural scenes has been divided into methods based on traditional stereo vision algorithms and methods based on deep learning.In this paper,in view of the sparse features of monocular endoscopic liver images and the inability to obtain true depth labels,a selfsupervised depth estimation method based on the combination of traditional SFM and deep learning is proposed.First,simulate the environment of the human abdomen,place the pig liver in the simulated environment,and collect images of different pig livers through monocular laparoscopy.The images collected by the monocular endoscope will be distorted,and the collected images need to be corrected.The traditional FSM method can only produce sparse or uneven reconstruction.This article first performs threshold transformation on the incremental SFM method(COLMAP),and then performs COLMAP semi-dense reconstruction to obtain the 3D point cloud and camera pose.Compared with dense SFM reconstruction and sparse SFM reconstruction,this method reduces a lot of The incorrect reconstruction of the points again guarantees the number of reconstructed 3D points.After that,the semi-dense reconstruction is subjected to projection transformation to obtain a semi-dense depth map,the semi-dense depth map is weighted,and some inaccurate 3D points are discarded or suppressed,and a weighted semi-dense depth map is obtained as the supervision data.Then,the weighted semi-dense depth map,camera pose,and liver image obtained in the image preprocessing stage are used as training data to participate in the training.In this paper,a weighted semi-dense depth map is used as the supervision data to guide the effective depth loss function to converge,and the mutual compensation depth difference function is constructed through the projection transformation of the two frames of the two-branch twin network.In the training network,considering the sparse texture of the pig liver image and the difficulty of feature extraction,the attention mechanism model with dynamic convolution is introduced into the FCDense Net network.The SKNet model allocates convolution kernels of different sizes or different receptive fields according to the input feature maps to realize dynamic extraction of feature information under different receptive fields.Since there are some invalid pixels in the weighted semi-dense depth map,in order to obtain a dense depth map,this paper proposes a depth difference loss function.In the same video sequence,the geometric constraints between two images with sufficient overlap can be used to compensate for the difference between the two depth maps.Finally,the liver data set was used to test the generated depth prediction model,and the feasibility and superiority of this method were proved through comparative experiments,and the effectiveness of the weighted semi-dense depth map and the fusion attention model network proposed in this paper was demonstrated through ablation experiments.