Research On Saliency Detection And Co-saliency Detection Based On Images Correlation

Image,as the objective reflection of natural scence,is the visual basis for human to understand the world.In recent years,with the widespread popularity of imaging equipment,such as digital cameras and smartphones,as well as the rapid development of the Internet,image has become the most common information carrier in today’s human social activities by virtue of its intuitive content,rich expression and convenient dissemination.Facing the explosive growth of image data,how to automatically locate the salient objects that attracts the attention of human eye in an image,namely saliency detection,have become the hot spots for scholars from home and abroad.However,after all,the content of one image is limited,and the detection performance of one single image is often restricted.Therefore,researchers begin to utilize the additional information provided by the similar images of the original image,to further improve the detection performance of the original image.In addition,in some actual scenes,the targets are no longer limited to one single image.Instead,we need to combine multiple images to analyze their mutual relation,and simultaneously operate a set of images with approximate objects to detect the co-salient objects,namely co-saliency detection.Image saliency detection and co-saliency detection are indispensable basic research topics in the field of computer vision.They have been widely used in many image processing tasks,such as semantic segmentation,image/video compressiong,object recognition and localization,with important theoretical significaticance and research value.In addition to the single image information,this thesis utilizes the correlation information between multiple images,to improve the performance of saliency detection and adapt to the detection of common salient objects in multi-image scence.The main innovations and contributions of this thesis include:(1)This thesis proposes a saliency fusion model via the use of similar images.Firstly,a group of similar images are retrieved according to the input image,and meanwhile,multiple saliency maps of the input image and similar images are generated by using existing saliency models.Secondly,based on the corresponding similarity between each similar image and the input image,an adaptive fusion method is designed to integrate such saliency maps,generating the saliency fusion map.Thirdly,an inter-image graph,for each pair of input image and similar image,is constructed to propagate the confident saliency values from similar image to the input image,yielding the saliency propagation map.Finally,the saliency fusion map and the saliency propagation map are integrated to obtain the final saliency map.Experimental results on two public datasets demonstrate the proposed model achieves the better saliency detection performance compared to the existing saliency models and saliency fusion models.(2)This thesis proposes a co-saliency detection model via the integration of multi-layer convolutional features and inter-image propagation.Firstly,the input image and its four co-images belonging to the same image category passes through the VGG16 network,to generate the multi-layer convolutional feature of these images.Secondly,by integrating the multi-layer convolutional features,we can obtain multi-scale synthesized feature maps,which contain both internal feature of the input image and correlative feature between all five images.Thirdly,with the help of low-level boundary features and high-level semantic features,the multi-scale synthesized feature maps are enhanced and the resultant multi-scale enhanced feature maps are fuesed to the initial co-saliency map.Finally,an inter-image saliency propagation method is utilized to refine the initial co-saliency map,yielding the final co-saliency map.The experiments are conducted on two public datasets.Comparing to many existing co-saliency detection models,the proposed model achieves the best performance.(3)This thesis proposes a co-saliency detection model using collaborative feature extraction and high-to-low feature integration.Firstly,the individual feature extraction module is used for the target image and its co-images,to obtain multi-level individual feature maps.Then,the well-designed collaborative feature extraction module which utilizes two feature extraction strategies,is applied to all highest-level individual feature maps,generating the collaborative feature map which contains the collaborative inter-image information.Finally,to balance the individual information and collaborative information,a high-to-low feature integration module is built to integrate the collaborative feature map and multi-level individual feature maps of the target image,yielding the co-saliency map of the target image.The proposed model achieves the consistently better co-saliency detection performance on both public datasets.The results approve that the model is effective.This thesis explores the correlation information between multiple images which has similar objects,and some advanced technical methods,such as non-negative quadratic programming,manifold ranking and convolutional neural network,to do research on two fields including image saliency detection and co-saliency detection.This thesis proposes a saliency fusion model and two co-saliency detection models,which consistently achieve the competitive detection performance and effectively promote the development of saliency detection,co-saliency detection,and other research fields.