Deep Learning-Based Co-saliency Detection Research

Co-saliency detection aims at discovering common and salient objects or regions from a group of relevant images.The difficulty lies in how to obtain adaptive feature representation with foreground consistency and foreground-background discrimination for different situations in different image groups,and how to model the inter-image inter?action to exploit the group context information for facilitating the co-saliency detection in each image.Although the existing researches on co-saliency detection have explored the above two problems extensively and deeply,there are still many obstacles to be solved urgently to achieve satisfactory co-saliency detection,especially it is still in the initial exploration stage to use deep learning tools for solving the problem of co-saliency detection.In this paper,three methods of co-saliency detection based on deep learning are proposed,which provide several new ideas for the research of co-saliency detection.The main contents and contributions are summarized as follows:1 A semi-supervised co-saliency detection algorithm based on Graph Attention Network is proposed.Firstly,multi-view feature selection is used to learn and fuse multi-view features of super-pixels.By exploring the complementarity of multi-view features,task-adaptive comprehensive feature expression is obtained.Then the initial graph topology structure is constructed on the optimized comprehensive features,and the interaction among the super-pixels in the group is modeled by the graph attention network,and the precise co-saliency reasoning is achieved by using the feature expres-sion with group context information.Finally,the multi-view feature selection network and the graph attention network are optimized sequentially by using the carefully de-signed cost functions as supervision.The proposed graph attention networks integrated graph optimization,feature learningand co-saliency detection inferring into a unified framework.It fully considers the intra-group superpixel interaction and the influence of group context information on feature learning.Impressive results of co-saliency detec-tion are obtained.2.A group semantic guided co-saliency detection framework is proposed.Firstly,a hierarchical low-rank bilinear pooling strategy is proposed to integrate the features of all the image in the group into a comprehensive group representation,and the co-category supervision are applied to make the group representation with abundant semantic in-formation.Secondly,under the guidance of group semantic features,we explore the complementarity between multi-level CNN features and accurate co-saliency maps are obtained.The proposed model explores the inter-image interaction at the semantic level for the first time,and the obtained group semantic information are exploited for the guid-ance of co-saliency detection in each image.The whole network structure is trained and optimized in an end-to-end manner,which improves the robustness of the model and the reasoning ability of co-saliency.3.On the basis of the second work,an improved and augmented deep learning network structure are proposed.We propose a pyramid attention module to emphasizes important image regions on multiple spatial scales and suppress background distrac-tions.In order to alleviate the semantic gap between group semantics and multi-scale vi-sual features,Semantic-Visual Feature Pyramid structure is proposed,which integrates group semantics and multi-scale visual features in a gradual manner.The former single-scale co-saliency supervision was replaced by multi-scale supervision,which facilities the multi-level feature learning and optimization.Based on the above three improve-ments,the performance of proposed co-saliency deep learning model has been further improved.