RGBT Saliency Object Detection Based On Collaborative Ranking Model

RGBT saliency detection based on visible light(RGB)and thermal infrared(T)images is an emerging hot research topic in the field of computer vision.Through adaptive fusion of visible light and thermal infrared data,RGBT saliency detection can deal with the challenges(such as low illumination,bad weather,etc.)in saliency detection based on single modality.In recent years,researchers have proposed RGBT saliency detection algorithms based on graph model and fusion of deep learning features,which greatly enhance the performance of saliency detection,but most of them do not fully exploit the essential relations between different modalities.This dissertation carry out relevant research on how to establish the essential connections between different modalities,and proposes a ranking model based on seeds optimization and the new collaborative ranking model in RGBT saliency detection.The main works of this dissertation include the following aspects:First,an optimal ranking model is proposed.In order to suppress the impact of challenging scenarios such as illumination variation and severe weather and take both the problem of noise in the image,the objects next to the boundary of an image into account at the same time,this dissertation proposes an RGBT saliency detection ranking model based on seeds optimization which is on the basis of traditional manifold ranking algorithm.First,the superpixel maps of the RGBT initial images are obtained by the superpixel segmentation algorithm,taking these superpixels as nodes,an appropriate graph model is constructed.Then introduces a cross-model consistency constrain to model the collaborative relations between different modalities.The saliency value can be calculated by an efficient optimization algorithm.Second,this dissertation introduces an intermediate variable to optimize the ranking seeds,and transforms it into a sparse learning problem to handle the problem of noise in the images.Finally,experimental results on the public benchmark datasets demonstrate the effectiveness of our proposed algorithm.Second,a collaborative ranking model is proposed.The traditional RGBT saliency detection methods only consider the collaboration between modalities but ignore the heterogeneity,and make it hard to fully exploit the complementarity between different modalities.Therefore,this dissertation proposes a new RGBT saliency detection method based on the collaborative ranking model,by making use of the collaboration and heterogeneity in different modalities,develops an efficient and accurate image saliency detection algorithm.Specifically,the superpixel maps of the initial images are obtained by the superpixel segmentation algorithm,and the graph model is constructed by using the appearance consistency and spatial continuity between the superpixel nodes,the nodes are composed of superpixels and the edges are determined by the incidence matrix.Then,this dissertation introduces the modality weight vector to construct the ranking model and achieves the adaptive fusion among different modalities.On this basis,we take both the collaboration and heterogeneity into account between different modalities,calculate the ranking function of multiple modalities,and allow the sparse inconsistency to reduce the influence of the heterogeneity,and formulate it as a sparse learning problem.An efficient optimization algorithm is designed to solve the proposed model.Finally,this dissertation provides a comparison platform for the RGBT saliency detection,compares the results of the proposed method with 15 popular methods and analyzes in details with the experimental results on two public RGBT datasets.The experimental results show that the proposed method can achieve accurate and efficient saliency detection performance.