Research On Dense Depth Map Generation Based On Edge Guidance

Due to the limitations of the collection equipment and the effects of the collection environment,it is still very difficult to obtain dense and accurate depth information in the scene.For example,when the surface of the object photographed by Kinect is smooth,there will be holes in the depth map.In addition,the depth information obtained by lidar in an outdoor environment is also very sparse.This thesis will use convolutional neural networks to complete sparse depth maps and generate dense depth maps.The main works done in this thesis are as follows:(1)This thesis proposed the concept of the edge-dist field and designed the edge-guided convolution layer.First,the depth edge map is converted into an edge-dist field according to the defined rules.Then,with the help of the edge-dist field,the edge-guided convolution layer can not only pay attention to the sparsity of the depth data and propagate the data confidence,but also keep the depth edges sharp during the depth map completion process and improve the performance.(2)This thesis designed a dense depth map generation network(EGCNN)based on edge-guided convolution,including a depth edge generation module,an edge-guided completion module,and a multi-scale refinement module.The depth edge generation module receives the high frequency components of the RGB image and the dense depth map after completing the sparse depth map using the existing algorithm as input.By combining the complementary edge information from them,a more accurate depth edge is generated.The edge-guided completion module uses the edge-guided convolution layer to extract depth information of different scales by inputting the sparse depth map,confidence,and edge-dist field to complete the sparse depth.The multi-scale refinement module processes the output of the edge-guided completion module,uses three cascaded hourglass networks based on standard convolution layers,and combines RGB information to further improve the output quality.(3)By comparing experiment results with sparse invariant convolution layer and confidence propagation convolution layer,the effectiveness of edge-guided convolution layer is verified.By comparing the completion results using different depth edges,the robustness of edge-guided convolution is verified.In addition,the experiment evaluated the influence of different edge-dist field definition rules on the performance of edge-guided completion module.The ablation experiment also shows the positive effect of the multi-scale refinement module on the overall network.At the same time,the comparative experiment with other algorithms proves that the model proposed in this thesis has strong generalization ability,and has better performance than other algorithms on multiple datasets and in different sparsity.(4)This thesis further applies the model proposed to the field of pedestrian detection.In practice,the edge-dist field is generated by using RGB image and original sparse depth information.With the guidance of edge-dist field and RGB image,the sparse depth map is completed to get dense depth map,which is combined with RGB image to get better pedestrian detection results.