Research On Image Co-saliency Detection Based On Mask R-CNN

Co-saliency detection is a method of extracting the same or similar saliency targets from multiple related images,which has become a very popular research topic in computer vision.The difficulty of co-saliency detection is how to distinguish the foreground and background in the picture group,and highlight the foreground target.And how to model the consistency relationship between the same group of pictures,and use group semantic information to improve the performance of co-saliency detection.This article does the following research on the above problems.To solve the problem that the objects with widely different semantic categories in the current co-saliency detection method are mistakenly detected as collaborative objects,this paper proposes a co-saliency detection algorithm based on convolutional neural network and semantic correlation CSCCD(CNN and Semantic Correlation based Co-saliency Detection).Firstly,the guided superpixel filtering method is used to process the superpixel area segmented by SLIC and the saliency results of the DSS model,and the clearer boundary outline is displayed.Then,using Mask R-CNN to extract the semantic features,define the semantic features and semantic consistency of the image,and propose the concept of the semantic related class of the image group.Based on this concept,an image group semantic association class is defined,which is used to model the semantic consistency between pictures in the same group,solve the semantic association problem of multiple images.Finally,the saliency detection area and the semantic consistency area of the image group are merged to obtain the co-saliency detection result.CSCCD innovatively uses the concept of semantic related class of image groups to solve the problem that objects with widely different semantic categories in the current method are mistakenly detected as collaborative objects.Since the current learning-based co-saliency detection method does not make full use of convolutional features,the generated collaborative saliency maps have edge blurring problems,MLF(Multi-Layer Fusion model),a collaborative saliency detection method based on Mask R-CNN is proposed.First,feature extraction isperformed on the input image and the images in its collaborative input group,then multi-scale feature maps are generated based on these feature maps,and the multi-layer feature maps of the input images are fused using feature pyramid networks to enhance the multi-scale feature maps.Next,the normalized and enhanced multi-scale feature maps are fused through a fusion strategy in the network to obtain the initial collaborative saliency map.the saliency map of the input image is obtained using a saliency propagation algorithm based on regional similarity.Finally,the saliency map of the input image and the initial co-saliency map are fused and normalized to obtain the final co-saliency map.MLF makes full use of the multi-layer convolutional features of deep networks,and obtains more semantic and edge information co-saliency maps.The experimental results on two co-saliency datasets,iCoseg and MSRC,show that the two models proposed in this paper can obtain good results on different indicators,illustrating the effectiveness of the model.