Salient Object Detection Based On RGB-D Information

Detecting and segmenting salient objects in natural scenes, also known as salient object detection, has attracted a lot of focused research in computer vision and has resulted in many applications. Most of the existing saliency detection methods exploiting color information and several priors to achieve good results. Although depth information plays an important role in the human vision system, it is not yet well-explored. Now people can get depth map easily thanks to the birth of a new generation of sensing technologies, such as the Kinect. So recently, several models taking depth into account for salient object detection but the performance is limited.In this thesis, top four 2D salient object detection methods are used to generate RGB image based saliency map. Then the results are fused with a depth-induced saliency map respectively to demonstrate how depth information makes a difference. Results show that depth information can really boost the performance. However, this method treats color and depth information independently, thus ignoring the strong complementarity between the appearance and depth correspondence cues. Inspired by this, two novel algorithms are proposed to detect salient regions with color and depth information.The first one is an optimization-based method. Firstly, the input color and depth images are abstracted into a set of regular super-pixels using an improved depth-aware over-segmentation algorithm. Secondly, an accurate background measure is computed by taking depth cue into account and further refined using an edge-preserving smoothing method. Finally, a foreground map through is obtained using a specialized multi-level RGB-D model, and a saliency map is computed through optimization.The second one is a learning-based method. New depth-induced region features are designed, and a random forest regressor is used to map rich features extracted from each region to a saliency score. The saliency map is constructed through multi-scale feature learning.The results of the two methods are complementary to each other. So at last the two saliency maps generated by the two methods are fused using Bayesian integration. Compared with previous methods, our saliency detection methods are much more effective and robust as demonstrated by experimental results.