Research On Several Key Issues Of Image Saliency Detection

Saliency detection is an important branch of image processing,which aims to locate the most attractive regions by human beings.Saliency detection not only can be used for analysing the content of images but also can be used to enhance the algorithms of other fields.Therefore,it is critical to research some methods for saliency detection.There exist some uniform characteristics in salient regions.The first one is that the salient regions are distinct in the entire image.Hence they can attract human beings' attention for the first time.The second one is that the salient region looks different because they have high contrast with their neighbour regions.Two types of saliency features are designed to simulate the above two characteristics: global saliency features are used to detect saliency from global view,and local saliency features are designed to detect saliency in a small local range.The core of this thesis is around the two saliency features.Global feature and local feature are designed for a traditional saliency method and a deep saliency method respectively,which are both utilized to detect the salient regions of input images.For the traditional saliency methods,they generally model the regions of images,and calculate the distance between different regions.Therefore,the model and the distance metric are very important for the final saliency results.To achieve robust model and distance metric,this thesis proposed to use a color fusion model with aggregated Wasserstein distance to design the global and local saliency features.For the deep saliency method,traditional convolution operations can catch the local information of feature maps but lack of global information.Therefore,this thesis proposes a self-attention deep network to dig the global information of feature maps.At last,this thesis brings the saliency detection to a specific application.The saliency information of image will be used to help network segment images under weekly-supervision.The detailed work and novelty of this thesis are shown as follows:1.This thesis proposes a novel saliency detection method based on aggregated Wasserstein distance.The traditional saliency methods generally rely on the information of the model and the accuracy of the distance metric.Existing algorithms usually model the image regions using the information of single color space,which may lose useful color information.This thesis proposes to utilize the Gaussian mixture model(GMM)to obtain more color information.Based on the Gaussian mixture model,this thesis proposes to use an aggregated Wasserstein distance,which has a closed-form solution for GMM,to calculate the visual difference between different regions precisely.And this thesis proposes a novel global saliency feature and local feature to generate saliency maps together.At last,spectral clustering is used to cluster their regions which have similar saliency features,and the results of clustering are used to filter the saliency maps.Experimental results demonstrate the effectiveness and reliability of our method.2.This thesis proposes a mutual self-attention network for saliency detection.Common saliency networks use convolution operations to extract deep saliency features.However,this kind of convolution operations only utilizes the local information of images,and lack of global saliency information.To resolve this problem,this thesis proposes to use the self-attention to obtain global saliency information.The self-attention can calculate the relationship of the pixels gloably during generating new feature maps.Therefore the network can absorb global information by a self-attention module.To improve the information of self-attention module,this thesis proposes a novel mutual self-attention module which uses mutual learning to improve the performance.Moreover,to recover the size of feature maps,this thesis proposes a novel channel-cross residual side outputs fusion module.The module fuses multiscale side outputs with the self-attention residuals to enhance saliency information.The experimental results between several saliency methods demonstrate the effectiveness of our method.And the ablation studies demonstrate the effects of the modules proposed in this thesis.3.This thesis proposes a saliency guided weakly-supervised image segmentation network.Deep learning usually needs lots of pixel-level labels to train the entire network,which is hard and expensive to achieve.On the contrast,image-level labels are much easier to obtain.This thesis proposes a network to generate pixel-level segmentation results using image-level labels.The network utilizes classification activation maps to obtain the location cues.But these location cues are too sparse and small.Therefore,this thesis proposes a novel saliency guided seeded regions growing method to expand these location cues and obtain more location information.Based on this,the network trains a full supervised segmentation network to generate pixel-level segmentations using image-level labels.Experimental results demonstrate that our network can achieve weakly-supervised image segmentation.