Manifold Ranking With Deep Convolutional Networks For Co-saliency Detection

Co-saliency detection is an emerging and rapid developing topic in computer vision area,which aims to detect the target sharing the similar appearances or common semantics in one image group,segmenting the pixel-wise co-salient objects.Specifically,manifold ranking constructs the correspondence between nodes with manual features,then regards those nodes with high confidence as reliable outputs.However,most available methods only ideally consider the appearance similarity instead of reflecting on the semantical and conceptual concurrency of those common foreground objects,leading to the inferior robustness of these traditional features under complicated realistic scenario.In this paper,we present a deep manifold ranking for co-saliency detection,two main contributions can be summarized as follows:(1)In this paper,we propose a hierarchical image co-saliency detection framework as a coarse to fine strategy to capture this pattern.We first propose a mask-guided fully convolutional network structure to generate the initial co-saliency detection result.The mask is used for background removal and it is learned from the high-level feature response maps of the pre-trained VGG-net output.We next propose a multi-scale label smoothing model to further refine the detection result.The proposed model jointly optimizes the label smoothness of pixels and superpixels.Experiment results on three popular image co-saliency detection benchmark datasets including i Coseg,MSRC and Cosal2015 demonstrate remarkable performance compared with the state-of-the-art methods.(2)In this paper,we explore its use in image co-saliency detection task,and present a framework that has three consecutive modules.Specifically,we first generate an initial detection result from the image convolutional features through unsupervised graph clustering.Afterwards,the post-processed output is taken as the pseudo labels of the second module that learns a neural network for more accurate sample affinity estimation,followed by a manifold ranking based label smoothing.Finally,with the smoothed labels as supervision,we design a saliency network that aggregates a hierarchy of multi-scale features to predict the co-saliency maps.Given one image as input,the proposed framework can be readily extended to image saliency object detection task.Extensive evaluations on both co-saliency and saliency detection benchmark datasets demonstrate that the proposed approach achieves favorable performance against a variety of supervised and unsupervised state-of-the-arts.