Image Saliency Detection Based On Multispectral Data Fusion

Image saliency detection is an important research area in the community of computer vision.In the era of intelligence,the accuracy and speed of multimedia information need to be solved.Saliency detection is to extract the most concise and useful visual information from images or videos,and is thus beneficial to the fast search for accurate positioning and the target retrieval.In recent years,with the development of image saliency detection technology,varieties of algorithms have been proposed for saliency detection.Although much progress has been made,it is still a very challenging problem in complex scenarios.In this dissertation,we propose two multispectral saliency detection algorithms based on collaborative graph model and multiscale deep neural network model respectively,in order to increase the accuracy and robustness of visual saliency detection in complex environments.The major works are summarized as follows:(1)How to construct the representation model to describe the similarity between two super-pixels according to multi-modal information,and then achieve robust image saliency detection?Visible light images are easily affected by light changes and bad environment,and thus bring a great challenge for the saliency detection task.Aiming at overcoming the limitations of single spectrum,we combines multi-spectral image information,i.e.,visible light and thermal infrared,to represent super-pixels collaboratively,and then propose a cooperative graph model to optimize the relationship between super-pixels.The robust image detection results are thereby are achieved.In particular,in order to exploit the intrinsic relationship between super-pixels,this dissertation uses the low rank and sparse representation model to dynamically learn the similarity relationship among super-pixels.At the same time,in order to make use of the more image clues,we combine more image cues,such as the spatial position of each super-pixel,with the learned similarity matrix to form the final similarity matrix.Then,we use the manifold ranking algorithm to obtain the saliency result for each modality.Finally,in order to achieve adaptive fusion of different modal information,a quality weight is introduced for each modality which can be learned from SVM.We combine the computed saliency maps with their quality weights to form the final saliency map.The experimental results on the dataset verify the effectiveness of the proposed method.(2)How to make full use of the deep features at different scales to depict the contrast measures of different sizes of images?Most of the existing saliency detection algorithms take detect target as a separate entity,and only use the global information to represent the appearance of the target.The intrinsic relationship is thus ignored among the global and the local parts of the image.This paper presents a multiscale saliency detection method based on multiscale deep neural network.First,the image is resized into three scales and we use three CNNs on these scales,then the feature extraction is transported to the network composed of fully connected layers network.In order to integrate a saliency map under different modes,we separately for each mode under the picture to train a SVM classifier,a significant picture for each mode of the input to the SVM classifier can produce a weight used to indicate the current mode of reliability.Finally,through a simple linear combination,the final map is obtained.