Sparse Depth Completion Based On Deep Learning

Aiming at the challenges and difficulties faced by the depth complementation of outdoor scenes,using a deep convolutional neural network with a powerful ability to express features as a tool,combining high-precision sparse depth in the autonomous driving scene and a large amount of data collected by other sensors,Research on improving the accuracy of dense depth maps after depth completion of outdoor scenes is a problem that needs to be solved at present.Research work related to sparse deep completion has been carried out for a long time,but most of the work has been performed in indoor environments.The sparse depth map in the outdoor environment is much sparse compared to the indoor environment,and the lighting,target occlusion,and depth of field also change more drastically than in the indoor environment.The accuracy of outdoor dense depth maps obtained using traditional methods is poor.how to obtain more accurate dense depth maps is still a technical difficulty in outdoor depth completion.This paper proposes a deep learning architecture that can generate more accurate dense depth for outdoor scenes based on a single image and sparse depth.The specific research content and related results are as follows:1.The surface normal vector can play a great role in generating accurate dense depth maps indoors,because the surface normal vector can represent the depth change method,which can produce high-quality dense depth through the integration of sparse depths,but outdoor The role of sparse depth completion in the environment has not been proven.Inspired by the work of indoor depth map completion,the network proposed in this paper estimates the surface normal to produce an intermediate representation of dense depth and can be trained end-to-end.2.Through the improved codec structure,the network proposed in this paper can effectively fuse dense color images and sparse LiDAR depth.Compared with the traditional codec structure,this structure can not only improve the performance on sparse depth completion,but also save network parameters.3.The occlusion problem is still a problem that has not been solved well in the research of outdoor sparse depth completion.In order to meet this challenge,the network proposed in this paper also predicts a confidence mask to deal with the foreground due to occlusion.hybrid LiDAR signal generated near the boundary.4.Combine the estimated values of the color image and surface normals with the learned attention map to improve the accuracy of the depth,especially for long distance areas.5.A large number of experiments prove that the model proposed in this paper improves the best performance on the KITTI deep completion benchmark test set.The ablation research also shows the positive impact of each model component on the final performance,and comprehensive analysis shows that the model proposed in this paper generalizes well to inputs with high sparsity or indoor scenes.