Research On 3D Display Algorithm Based On Binocular Endoscope Image

3D films based on the disparity are widely used in recent years,its principle of endoscope image three-dimensional display is accurate in order to let the doctor to "see" patients with lesions,specific depth perception of information convenience of operation,shorten the operation process.The smoke in endoscope image seriously affect vision and disparity estimation,with the rapid development of deep learning technology,the image desmoking and disparity estimation models based on it show great advantages in accuracy and speed,and gradually replace the traditional physical models.this thesis carries out desmoking,disparity estimation and 3D display for endoscopic images based on the deep learning model.Firstly,in view of the problem that the endoscopic image set does not contain real smoke labels,we adopts Blender to add smoke to the image as the training set for supervisied training.We use U-Net as the basic model.In order to retain the image details and color information in the model,the training images through Laplace pyramid transformation are added into each layer in encoder.For the sake of better intermediate features,lightweight CBAM attention module is added into the last five layers of the decoder.The optimal parameters of each layer of the network were obtained by sending the training set into the model and training.The peak noise ratio index was 31.05 d B,the structural similarity index was 0.98,and the processing time of a single image was90.19 fps,which could desmoking the endoscopic image in real time.It can also help purify the doctor’s field of vision.Secondly,in view of the problem that binocular images do not contain real disparity,we take advantage of the disparity conversion relationship between binocular images,and combine the left eye image with the disparity to obtain a virtual right eye image as a label for self-supervised training.In order to improve the performance of the model,HS-Resnet is selected as the encoder.HS-Resnet performs multi-scale segmentation and concatenate in the process of feature extraction,so that the network can effectively extract features of various receptive fields at different scales.In order to make the generated disparity complete and smooth,we extract the disparity at different scale levels of the decoder.In this thesis,SSIM is 0.8826±0.0678,PSNR is 17.2594±1.6254 on laparoscopic test set,All these indicators have been significantly improved.Finally,because the endoscopic image disparity does not conform to the observation rules of human eyes,we convert endoscopic disparity to human eye disparity,and then studies a three-dimensional vision aid scheme based on chromatic aberration.The red component of the original image is separated and combined with the disparity,and then fused with the blue-green component to obtain a composite image.As long as you wear red and blue glasses,you can observe the three-dimensional display results of endoscopic images.