Image Saliency Detection Based On Deep Learning

Saliency detection is the process of extracting significant target regions in the image by simulating the human visual attention mechanism.As a preprocessing step in many computer vision applications,saliency detection has been widely used in a variety of tasks.Based on summarizing and analyzing the state-of-the-art about saliency detection in color images and hyperspectral images,this thesis proposes and implements two algorithms about color image saliency detection via adversarial learning and one algorithm about hyperspectral image saliency detection via self-supervised learning by means of deep learning.1.Color Image Saliency Detection via Two-Stream Feature Fusion and Adversarial Learning(Sa TSAL)Under the learning framework of CGAN,an algorithm of color image saliency detection via dual-stream feature fusion and adversarial learning(Sa TSAL)is proposed.The algorithm uses VGG-16 and Res2Net-50 as the dual-stream heterogeneous backbone network for saliency detection,and realizes multi-level,fine-grained,and rich-scale image feature extraction from low-level to high-level.In each single-stream structure,one multi-channel feature map processing step based on convolution tower module is introduced,which further enriches the multi-scale information of the same level feature map in each feature extraction path.The saliency map is generated by top-down,level-wise fusing of cross-stream features laterally,which effectively combines the large-scale context information of target saliency with the small-scale features of saliency target boundary.The comparison experiments with other ten excellent saliency detection algorithms performed on four public datasets including ECSSD,PASCAL-S,DUT-OMRON and DUTS-test demonstrate the effectiveness of the proposed Sa TSAL algorithm.2.Color Image Saliency Detection via Two-pooling Enhancement Module and Adversarial Learning(SaTPEAL)Under the learning framework of CGAN,a saliency detection algorithm based on dual-pool,enhancement modules and adversarial learning(SaTPEAL)is proposed.By combining the backbone network based on Res2Net-50 with global second-order pooling module and band pooling module respectively,a dual-path feature extraction mode based on two-pooling strategy is constructed,which can effectively promote the embedding different levels of high-order statistical characteristics from color input image into each level of feature map and the capture of remote spatial dependence between different components from salient objects.By laterally level-wise fusing of different levels of feature maps from top to bottom,a rough saliency map is generated,which makes full use of the high-level semantic context and low-level features of salient objects.Furtherly,a post-processing module based on iterative residual deconvolution is constructed to refine and enhance the rough saliency map,and the final saliency map with clearer boundary and more complete structure can be generated.End-to-end detection scheme makes SaTPEAL more robust.The comparison experiments with other ten excellent saliency detection algorithms performed on four public datasets including ECSSD,PASCAL-S,DUT-OMRON and DUTS-test demonstrate the effectiveness of the proposed SaTPEAL algorithm.3.Hyperspectral Image Saliency Detection via Joint Attention Mechanism and Unsupervised Segmentation(HSa JAMUS)In this proposed algorithm,the deep feature extraction network is constructed by introducing channel-spatial joint attention module into the Res2Net-50 based backbone network,and used to extract multi-channel feature maps combining of spectral-domain and spatial-domain information from hyperspectral input image.By applying pixel-wise classification on multi-channel feature maps based on channel-domain maximum-value,the channel label can be predicted for each pixel.Then,the input hyperspectral image is over-segmented via SLIC(Simple Linear Iterative Clustering)algorithm,and clustering correction is performed based on the super-pixel segmentation results in order to generate guiding label.Finally,by minimizing cross entropy loss between guiding labels and predicted labels,the self-supervised learning of the deep feature extraction network is realized.At the same time,the multi-channel feature map is over-segmented into super-pixels,and a saliency map can be generated based on two-stage manifold sorting.The self-supervised learning and saliency map generation are carried out alternately and continuously optimized.Finally,the achieved multi-channel feature map obtained can effectively combine the joint saliency characteristics from spectral domain and spatial domain about the hyperspectral input image,and the saliency prediction results tend to be stable.The comparison experiments with other seven hyperspectral image saliency detection algorithms performed on HS-SOD dataset demonstrate the effectiveness of the proposed HSaJAMUS algorithm.