Research On RGB-D Image Visual Saliency Detection Based On Feature Fusion

Visual saliency detection can be defined as detection for objects of interest with computational model applied which simulates the human visual attention mechanism.Research shows that visual saliency is closely related to the depth information of the visual scene,so visual saliency detection has become a hot topic in the field of 3D perception.Human visual system has excellent 3D visual perception capabilities and can quickly locate objects of interest in the scene through visual attention mechanisms.The hypothesis based on the attention of object considers that semantic objects directly attract attention because their semantic category features have visual saliency.3D sensors are widely used to improve the ability of intelligent robots to sense objects in the real environment.The RGB-D images of 3D sensing data provide complementary constraints of color information and depth information to better describe the semantic category features of objects in the scene.Therefore,the detection of visual saliency of RGB-D images based on semantic categories has important theoretical and engineering significance for intelligent robots to scene perception.Taking the visual saliency of RGB-D images as the research object,this paper,in aim to raise F-measure in saliency detection model and make detection results better conform to human visual perception,explores the intrinsic link between the visual saliency detection of RGB-D images and the human visual attention mechanism from the perspective of probability statistics.Specifically,the study focuses on the problems of low accurarcy and recal rate in existing methods for salient feature extraction of RGB-D images,the fusion of salient features,and introduction of prior knowledge,which mainly includes:(1)The extraction method of salient features of Depth image based on semantic categories: the present artificial salient features of Depth image fail to dig up depth information effectively,especially the semantic category features hidden in scene structure information,and the object area with salient semantics thus cannot be highlighted as a whole in the condition of low depth feature contrast.Therefore we separately extracted the global context semantic features of Depth image and RGB image through deep convolutional neural networks,i.e.Clarifai networks,and established semantic category links between them,and then presented the salient features of both RGB image and Depth image in the semantic space.By computing visual saliency based on salient semantic category features of RGB-D image which are extracted by two Clarifai networks,the differences in semantic categories of salient objects are reflected in a way that is consistent with human visual perception.(2)The fusion method of salient semantic category features with feature relativity of RGB image and Depth image taken into account: the links between salient features of RGB image and Depth image should be considered when we fuse salient features of RGB-D image.Assuming that artificial salient features are linear or non-linear,the existing fusion method of salient features of RGB-D image fails to explore the relation among salient semantic category features.By analyzing the distribution of 3D visual saliency in RGB-D image,we applied Class-conditional Mutual Information to measure the relevance among salient semantic category features extracted by Clarifai network.On the assumption that salient semantic category features conform to conditional independent distribution,we used Bayesian framework to fuse salient features and obtain the likelihood of RGB-D image saliency.The fusion of salient semantic category features based on Bayesian framework can avoid the interferences from feature fusion of addition and multiplication,which reflects better robustness.(3)The estimation method of visual saliency of RGB-D image with prior semantic categories combined: based on prior depth information,the existing detection method for visual saliency of RGB-D image can highlight salient objects,but fails to restrain background areas effectively.Analyzing the datasets of human visual saliency from the perspective of probability statistics,this paper describes the prior weightings of salient semantic category features based on Dirichlet-Polynomial prior distribution and learns the prior distribution incrementally.On account of the fact that the prior distribution of semantic category saliency is in Dirichlet-Polynomial distribution,we applied revised discriminative mixed-membership naive Bayes(DMNB)to generate a model to compute the posterior probability of RGB-D image visual saliency and examine the datasets in small sample,which solves the problem of low generalization in depth prior feature-based computing methods for visual saliency and obtains higher F-measure in the benchmark datasets of visual saliency detection of RGBD image.