| Salient object detection aims to simulate the mechanism of human visual attention and accurately locates the most attractive regions in images.Due to the development of depth camera,depth map has been used to detect salient objects and achieved good results.Depth maps provide rich stereo information for salient object detection and enhance the model’s ability to distinguish foreground and background.However,the depth sensor is easy to be interfered by environmental factors when capturing depth map,and the quality of depth is uneven.The stereo information provided by the depth map of poor quality is not reliable,so the depth confidence gradually attracts attention.Some researchers consider generating new depth maps through depth estimation to reduce the noise interference of the low quality depth.In view of the above problems,the main work of this paper is summarized as follows:Depth confidence measurement algorithm for salient object detection.In this method,depth estimation network and depth discrimination network are used to calculate depth confidence,which use a weight to measure the reliability of spatial information in original depth.To be specific,depth is firstly inputted into U-Net network,and the difference between predicted map and groud truth is compared.The smaller the difference is,the higher the depth quality is,so as to screen out high-quality depth maps.Secondly,high-quality depth maps and its corresponding RGB maps are used as training sets to generate new depth maps under the action of depth estimation network.This method is task-driven to calibrate the original depth and better express the scene layout.Since some of the original depth still contains useful clues,the method uses depth identification network to calculate the confidence of the original depth.Finally,the depth confidence is used to fuse original depth and estimated depth,and then,the enhanced depth map is generated.This method makes reasonable use of the advantages of original depth and estimated depth,improves the overall quality of depth maps,and reduces the influence of low quality depth maps to a certain extent.On this basis,in order to effectively fuse RGB and enhanced depth,two multi-mode fusion algorithms are proposed:(1)RGBD salient object detection algorithm based on deep guidance.This method is guided by depth to enable RGB to locate significant areas.To be specific,in order to better play the auxiliary role of spatial information in the depth map,this method uses depth to generate attention map,and integrates it with RGB features across modes to reduce inconsistency between the two modes and effectively integrate cross-mode complementary information.(2)Multi-layer dynamic fusion RGBD salient object detection algorithm based on convolution pyramid.In this method,the features of each layer are gradually fused,and each layer is given a different weight,while the weight changes dynamically in the training process.Specifically,this method uses VGG16 to extract RGB and depth features.Secondly,the convolution pyramid module is introduced to obtain significant regional information at each scale by multi-scale grouping convolution.Then in the decoding stage,stripe pooling module is used to obtain the relationship between front and background in the RGB and depth features,and effectively distinguish foreground and background;Finally,the dynamic fusion of high-level semantic information and low-level detail information gives full play to the role of locating significant areas.In order to verify the effectiveness of the proposed scheme,the enhanced depth is applied to two multi-mode fusion algorithms and tested on public data sets.Experimental results show that the proposed method achieves good performance. |
