Research And Application On RGB-D Salient Object Detection

Visual saliency has been a fundamental problem in neuroscience,psychology,and vision perception for a long time.It refers to the measurement of low-level stimuli that grab human attention in the early stage of visual processing.We witness that the computation of saliency is originally a task of predicting where people look at an image,and recently has been extended to object-level saliency detection that involves separating the most conspicuous object from the background.This work focuses on the object-level saliency modeling,which benefits various applications including object detection and recognition,content based image retrieval,object aware image thumbnailing,etc.2D Saliency:For saliency detection on 2D RGB image,most existing algorithms can be roughly divided into two categories,i.e.,local and global.Local approaches detect salient objects by measuring the rarity of a particular image region with respect to its neighborhoods.Itti et al.first propose an influential saliency computational model,which performs center-surrounding differences on feature maps to obtain the local maxima of stimuli.Harel et al.define a graph on image and adopt random walks to compute saliency.To highlight the whole salient object,multi-scale contrast and multi-cues integration techniques are used.Due to lacking of global relationsand structure,local contrast methods are sensitive to high frequency content or noises.Global methods estimate saliency of a region based on its holistic rarity from an image.the Achanta define saliency by computing color difference from the mean image color on pixel level.Yet,this definition only accounts for first order average color and easily results in degraded performance on cluttered scenes.Goferman et al.propose an improved method that highlights salient objects with their contexts in terms of low-level clues and global relationships.3D Saliency:Contrary to the significant progress in 2D saliency research,the work leveraging depth information for saliency analysis is a bit limited.Niu etal.[33]exploit binocular images to estimate a disparity map and only use depth data to identify salient objects.So the performance is highly dependent on the quality of disparity map estimation which is another classical and challenging Computer vision problem.Later,Lang et al.[25]conduct a comparative study of eye fixation prediction,rather than salient object detection,in 2D and 3D scenes after collecting a pool of 600 2D-vs-3D image pairs.Computer vision is more and more attention in recent years,the depth information which has 3D layout and shape features from depth images proved to be more informative.with the development of sensor technology,it can capture the scene simultaneously RGB color image and depth information,with the help of 3D image RGB-D for effectively saliency detection has gradually become the hotspot of computer vision research.However,most saliency detection methods based on 2D images do not make good use of depth Information.In this paper,an effective saliency detection method for RGB-D image is presented.It extracts color feature combined with depth saliency feature and detects salient objects based on photographic composition prior and background prior.First,original depth map is preprocessed to form depth saliency feature by background vertex area,photographic composition intersections,and compactness method.Then the association matrix is constructed by the adjacency weight of comprehensive feature.Manifold ranking is running from foreground view to form foreground saliency map based on photographic composition prior and fusion of depth saliency feature and color feature.In order to correct the error caused by assumption,the boundary connectivity is used to suppress background from background view.Final saliency map builds on fusion of foreground and background saliency map.Then we propose a method of depth saliency map which could enhance RGB-D saliency detection.This method firstly uses the method of LBE local background enclosure for depth map detection,and to be depth saliency feature strengthen RGB-D images of RGB maps through manifold ranking to figure out RGB saliency map;then using RGB saliency map to guide the depth map through manifold ranking to get depth saliency map;finally saliency map builds on fusion of RGB saliency map and depth saliency map.A comparative experiment was carried out on two public database,RGB-D1000 and NJU-DS2000.The experiment shows that the two methods we proposed can significantly increase the detection effect.In the end,we proposed a method of RGB-D image saliency detection with depth saliency features,which is applied to face detection.This method is using the RGB-D saliency detection which proposed in the first part to get the saliency map as the input map for the face detection to get the target.The RGB-D face image database is collected manually,and the original color map is compared with the original RGB images for face detection.The experimental results show that the accuracy of target detection and the number of false detection can be improved.