Two-stream Encoder Generative Adversarial Network For The Method Research Of Co-salient Detection

Co-saliency detection is a new branch of the computer vision,and it can detect common salient objects in a group of relevant images by simulating the human visual attention mechanism.At present,it has been widely used in various computer vision tasks,such as collaborative segmentation,target detection and other fields.Co-salient methods based on deep learning are mainstream now,which have achieved a good detection effect,but there are still two kinds of problems.On the one hand,the published co-saliency datasets have small training sample size,which is easy to make model over-fitting;On the other hand,the existing methods fail to effectively represent the semantic consistency among a group of related images,thus limiting the inference ability of the models for common salient objects.Aiming at the above two kinds of problems,the paper proposed a co-salient detection method based on the two-stream encoder generative adversarial network(TSE-GAN).Its training was divided into two training stages,and the main work is as follows.In the first stage of training,the sub-modules of the TSE-GAN were pre-trained.In the one hand,the published single saliency datasets were adopted to pre-trained the saliency object detection generative adversarial network(SOD-GAN),the way not only alleviated the problem of insufficient co-salient label samples but also made the module extract Intra-saliency features;On the other hand,the classification network(CN)was pre-trained by using the category label of the co-saliency dataset,the module could extract category semantic features,at the same time,it could paved the way for the group-wise semantic encoder(GS-Encoder)to acquire inter-saliency features with semantic consistency in the next stage of training.In the second stage of training,The TSE-GAN was based on the sub-modules of the first stage,and it was trained on the co-salient datasets.In the aspect of whole model construction,the TSE-GAN effectively solved the fusion of intra-saliency and inter-saliency features of twosteam encoders.In addition,the paper put forward the multi-scale semantic fusion network(MSFN),which integrated multi-scale group-wise semantic features to obtain the inter-saliency features with the semantic consistency,so as to solve the poor semantic consistency in a group of related images.Finally,the proposed method was performed ablation experiments on the i Coseg and Cosal2015 datasets,and the effectiveness of each branch was proved;our method was compared with 13 popular algorithms,the superiority of our method is verified.