A Study On Image Co-saliency Detection

Inspired by the human visual attention mechanism,in the field of computer vision,the study of image saliency aims to make the computer automatically select out the salient objects from the image.In recent years,the scale of multimedia data represented by digital image has an explosive growth,and how to effectively find out the common salient object from the large-scale image collections has become an urgent problem.Compared to the detection of independent salient object in a single image,cross-image co-saliency detection aims to discover the common and salient foregrounds from the relevant images,while within-image co-saliency detection aims to identify the common salient objects within an image.Although many algorithms based on learning salient visual features and designing computational framework have been proposed,different reasons lead to the unsatisfactory performance of co-saliency detection.On one hand,the increasing complexity of image scene brings huge challenge for detection.On the other hand,as a new research direction,co-saliency detection still needs further study in theory and algorithm.The main contribution of this paper is the following three-folds:For cross-image co-saliency detection,this paper proposes a new co-saliency detection framework via the fusion of multi-view features,which explores the complementarity between different types of visual features and optimizes the contribution of them in co-saliency detection.It reasonably leverages the multiple types of features and improves the detection performance.To be specific,we firstly generate the high-quality saliency map of each image by deep learning technique,and then we get the superpixel collections and extract the multi-type visual features of them and establish a multi-graph model with superpixels as the vertices.In the following,we jointly optimize the co-saliency value and the importance of multi-type visual features of superpixels to obtain preliminary detection results.Finally,the single saliency map and the preliminary co-saliency map are integrated with an adaptive rank constraint algorithm to further enhance the detection performance.In addition,we extend this work to the co-saliency detection of RGB-D images with just fusion the depth information and achieve state-of-art results.For within-image co-saliency detection,a novel co-saliency detection method based on multi-scale superpixels is proposed in this paper,which can better capture the visual representation of the salient objects in complex scenes.Specially,we firstly detect the foreground area of the image based on the saliency map of a single image,and then we segment it into multi-scales superpixels and successively calculate the co-saliency probability with the feature contrast at different scales to discover the co-salient objects.In the following,the initional co-saliency maps are intergated and the angular embedding algorithm is brought to refine the raw maps with the utilization of image boundary,which reserves the co-salient object while deletes the uncommon area simultaneously.Finally,we evaluate the proposed mathods on cross-image co-saliency detection datasets and within-image co-saliency detection dataset.Besides the traditional pixel-level saliency evaluation methods,i.e.Precision-Recall curve,F-Measure,AP score and Mean Absolute Error,the structure similarity measure based on image structure information is also introduced in this paper,the results demonstrate the superiority of our proposed methods both subjectively and objectively.