Research On Saliency Detection Of RGB-D Image

When the human vision system perceives the surrounding scene,the information obtained not only includes the RGB information generated by the object reflecting different spectra,but also includes the depth information provided by the stereo vision formed by two eyes.The wide applications of RGB-D sensors can improve the ability of computer to perceive objects in the real environment,and the high availability of depth sensor provides valuable supplementary information for the modeling and applications of RGB-D image processing.Based on the depth related visual characteristics of human visual system,this paper discusses the mechanism of visual attention in stereo vision.Aiming at the two task scenarios of visual saliency prediction and saliency target detection of rgb-d image,this paper focuses on the deep saliency feature expression and multi-modal feature fusion in the saliency detection of RGB-D image.The research content of this paper includes the following three aspects:1)The mechanism of depth feature's influence on visual attention in stereo vision is studied.A depth saliency feature based on disparity tuning mechanism and an RGB-D viusal saliency prediction algorithm are proposed.According to the parallax selection characteristics of the visual nerve in MT area of the cerebral cortex,the neural excitation response is modeled as a depth feature to predict visual saliency.This method explains the effect of depth feature on visual attention from the perspective of physiology.In the three largest visual saliency prediction datasets for RGB-D images,the result of this method is better than that of the existing methods.2)A two-stream refinement network for RGB-D salient object detection is proposed.The fusion based refinement network(FRN)and propagation based refinement network(PRN)are designed.The function of FRN is to fuse the multi-modal RGB and depth features as well as the multi scale intermediate convolution features in manner of intermediate fusion.As a result,the internal correlation between the RGB and depth map features is mined,and the high-resolution shallow convolution features are adopted to refine the boundary of the detected object.The function of PRN is to use a shallow network to learn the affinity matrix of each pixel,and further refine the result of preliminary detection.3)The fusion method of RGB and depth features in stereo vision attention is studied.An attentive cross modal feature-fusion method and an RGB-D image salient object detection algorithm based on deep neural network are proposed.This method uses the idea of residual connection,introduces the attention guidance mechanism,and realizes the optimization and efficient fusion of multimodal features.This method has achieved the best results on the eight public datasets for salient object detection.