A Study Of RGB-Guided Sparse Depth Completion

Dense depth maps are critical in areas such as photogrammetry,robotics,and autonomous driving,and depth complementation is a popular research direction in depth prediction due to its accuracy and affordability.Existing depth completion methods use a combination of RGB and sparse depth information for input,but existing networks often pursue accuracy and design huge network structures,which are not conducive to deployment and may have low robustness.In this paper,we explore the above problems in terms of lightweight and robustness,firstly we design a network structure according to the data feature and the characteristics of depth complementation in terms of lightweight,and strike a balance between parameter number and accuracy.Select post-fusion and design the corresponding encoder for RGB and sparse depth,use less parameters to achieve a structure with strong capacity.Secondly,the impact of the two network components on robustness is explored.Most works use channel concatenation or element summation to simply fuse the information of the two modalities.In this paper,we propose a dynamic gated fusion module that combines the information of two modalities with a sparse distribution of the input depth as a guide and combines the two modality features more efficiently by dynamically generating fusion weights.Further to alleviate the loss of spatial information caused by downsample operator,a spatial implicit convolution block is proposed to replace it,and an upsample operator is designed correspondingly in decoder.The spatial geometric features are encoded by converting spatial information into channel information for compression through sub-pixel technique.The experimental results demonstrate the effectiveness and improvements of the proposed modules.The proposed network achieves advanced results on two challenging public datasets:KITTI Depth Completion and NYU depth v2,only using a small number of parameters.Achieving an excellent balance of performance and speed.Good performance is also achieved in robustness tests on scenarios outside the training distribution,extreme illumination not seen in DDAD and RobotCar,and sparse number of input depth points.The end-side deployment tests demonstrate that the network in this paper can perform depth prediction in real time.