Depth Completion With Confidence Propagation,Semantic Information And Feature Augmentation

Scene depth completion is an important topic in computer vision,which aims to obtain accurate dense depth map from sparse depth map.The obtained depth information plays a key role in the fields of robotics,autonomous driving and augmented reality.Neural network greatly promotes the development of depth completion with its powerful feature expression ability and end-to-end training method.In this paper,convolutional neural network based depth completion is studied form three aspects,namely,feature extraction of sparse depth map,joint depth completion and semantic segmentation,and optimization of joint network.The main contributions are as follows:(1)A small depth completion network based on confidence propagation is built,consisting of the confidence propagation module,the scene understanding module,the depth optimization module and the fusion tuning module.The confidence propagation module explicitly considers the certainty of input signal,solving the problem that the depth map collected by Li DAR is highly sparse and irregularly spaced.The fusion tuning module fuses depth information from the depth map and scene information from the color image to solve the problem that the collected depth map is lacking in scene information,and realizes depth completion from coarse to fine.(2)By exploring the intrinsic relationship between depth completion and semantic segmentation,a depth completion method based on semantic information is proposed.A joint network based on codec structure is built,which explicitly improves the network's ability to extract scene structure information.Moreover,a cross-domain edge consistency loss is proposed,which uses the ground truth semantic segmentation map to guide the completion of depth map,and solves the problem of blurred edges.(3)By optimizing the joint network,a joint network based on feature augmentation is built,consisting of the encoding module,the feature deinterference module and the interactive decoding module.The feature deinterference module splits and recombines the shared feature in the network to solve the information interference problem in the shared feature.The interactive decoding module realizes the information interaction in the decoding phase,including unidirectional guidance of semantic segmentation to depth completion and bidirectional interaction between joint tasks.