| Accurate and complete depth value information is crucial in practical applications and can be widely used in important computer vision tasks such as autonomous driving,3D reconstruction and robotics.In order to obtain high-quality depth data of the surrounding environment,the raw depth images that people obtain through passive depth sensor systems are relatively influenced by environmental factors,and the accuracy of the resulting depth values cannot be applied to practical tasks.Although active depth ranging sensing systems such as TOF cameras and LIDAR,which emit signals actively to obtain depth information,can well overcome the drawback that passive depth sensors are not affected by environmental factors and the results are more accurate,the depth images obtained still have problems such as missing edges,extremely sparse and irregular depth pixel values,which are also difficult to meet the needs of practical application tasks.Thanks to the powerful characterization ability of CNN,the research of depth complementation algorithm based on deep learning has led the development trend,and more dense depth images can be obtained by the powerful network model.However,in the actual task,besides requiring high accuracy,we also want the algorithm to have excellent performance in speed,so how to design a lightweight and high accuracy depth image complementation network is an urgent problem to be solved.Based on deep learning algorithms,this paper carefully studies and analyzes the current mainstream multimodal fusion depth complementation algorithms based on RGB images and sparse depth maps,proposes a strategy to gradually fuse RGB images with sparse depth images and designs a powerful fusion module to help them fully fuse.Specifically,the algorithm consists of a PAN branch that mainly recovers spatial resolution,an FPN branch that mainly recovers depth resolution,and a fusion branch that gradually guides the recovery.important depth information in the depth image;the fusion network starts from the bottom layer and gradually fuses the multiple feature information extracted from the two branches to recover the image resolution.We encourage each fusion layer to approach the labeled image at the corresponding resolution,and thus adopt a multi-level supervision approach to improve the depthcompletion effect.Finally,the effectiveness of the network model algorithm proposed in this paper and the rationality of the model structure design are verified by experimental analysis on the outdoor open source KITTI depth dataset. |
