Research On Multi-modal Image Saliency Detection Algorithm Based On Multi-graph Learning

With the progress and development of modern multimedia image processing technology and the rapid growth of image data,how to extract the interesting and valuable information from the huge image data efficiently and effectively filter out the redundant data has become an urgent problem in the field of image processing and computer vision.In recent years,as one of the important branches of computer vision,visual saliency detection can assist and improve a series of visual processing tasks.It has been widely used in many fields,such as scene classification,visual tracking,target redirection,semantic segmentation and so on.It has already become a hot research subject.Although the research of single-mode image saliency detection algorithm has reached a very high level in specific test data sets and simple scenes,due to the lack of generalization of the algorithm,the saliency detection accuracy in complex scenes such as poor lighting conditions,bad weather,similar color and image noise still needs to be improved.In recent years,due to the continuous maturity of various imaging technologies,more and more data are obtained from other types of spectral imaging(thermal infrared image is not sensitive to light),which makes researchers in related fields try to integrate multiple different but complementary multi-modal information,such as visible light and thermal infrared information(RGB-T),which improves the accuracy and robustness of saliency detection to a certain extent.Therefore,this paper will focus on image saliency detection and multi-modal saliency detection.Specifically,our main work is as follows:(1)The traditional research on saliency is based on the division of image structure and the construction of graph model,and then on the basis of graph model,the research on the smoothness and distribution of saliency is carried out.Most of the existing image saliency detection algorithms use two prior information: background prior and boundary prior to assist other saliency clues(such as contrast)to achieve good saliency detection results.However,their use of the boundary prior is fragile and simple,and their fusion with other clues is basically heuristic.In order to solve this problem,this paper introduces a background measurement method considering boundary connectivity based on graph model to improve the robustness and accuracy of the model.This method can describe the spatial layout of image region compared with image boundary intuitively,and integrate it with the classical popular saliency detection model of ranking to integrate a whole saliency optimization framework,and then to many low-level cue and background measurement are integrated to get more robust and accurate detection results.Then we use different benchmark data sets to carry out experiments and comparative analysis.The experimental results show that our saliency detection algorithm is effective.(2)Images with a single mode cannot cope with image saliency detection in complex scenes such as low light and bad weather,so we consider integrating image information of multiple modes to improve the situation.However,the most of the existing image saliency detection algorithms based on graph model rely on a single graph model.For a variety of modes,it is obvious that multiple graph structures need to be acquired for parallel learning.Therefore,this paper will also study the multi-modal image saliency detection based on multi graph learning.Specifically,from the point of view of multi-graph learning,this paper uses the data of visible and thermal infrared spectra to carry out multi-graph learning,and uses Markov metric to fuse different spectral data to build a joint multi-modal graph model.Then,we combine it with the popular image saliency detection model to build a saliency optimization framework.For multi-modal images,we can extract the saliency features of different modes cooperatively and get better saliency detection results by adaptive fusion.Through the comparative experiments and results analysis on two multi-modal datasets,it is concluded that the joint multi-modal graph model has a saliency improvement in the accuracy of image saliency detection compared with the single-mode graph model.