RGB-T Image Saliency Detection Via Collaborative Graph Learning

Image saliency detection is a research hotspot in the field of computer vision and multimedia.Fusing complementary RGB and thermal infrared data has been successfully applied to many computer vision tasks,such as object tracking and person re-identification.RGB-Thermal(RGB-T)saliency detection aims to use thermal infrared image to assist salient object detection with visible image.But there are big objects without interior appearance consistency,objects next to the boundary of an image,noises caused by image registration near boundaries of RGB-T images in the image.For these challenges,existing RGB-T saliency detection methods use traditional graph models,but fixed graphs only includes local information and ignores the inherent intrinsic relations between nodes,resulting in poor performance of capturing the appearance information of the image well,there are still some shortcomings.We research on the above problems,construct a new RGB-T image saliency detection dataset,and propose two RGB-T saliency detection models based on collaborative graph learning.First,contribute a more challenging benchmark dataset for the purpose of RGB-T image saliency detection.Due to the sparseness of the current RGB-T image saliency detection dataset,in order to better test the robustness of the proposed model,we consider the many kinds of complex challenge factors,construct a RGB-T image saliency detection dataset containing 1000 pairs of spatially aligned RGB-T image pairs and their ground truth annotations.Second,multi-modal multi-scale noise-insensitive ranking(M3S-NIR)algorithm based on multi-scale collaborative graph learning is proposed for RGB-T image saliency detection.First of all,we segment spatially aligned RGB and thermal images together into a set of multi-scale superpixels.Then,we take these superpixels as graph nodes and performs multi-modal multi-scale manifold ranking to achieve saliency calculation,in which the cross-modal and cross-scale collaborations are performed to integrate different kinds ofinformation.To handle noises and corruptions of ranking seeds(i.e.,boundary superpixels)introduced by salient objects and RGB-T alignment,we introduce an intermediate variable to infer the optimal ranking seeds,and formulates it as a sparse learning problem.Finally,we use a unified alternating direction method of multipliers(ADMM)based optimization framework to solve the ranking model efficiently.Extensive experiments on the benchmark dataset demonstrate the effectiveness of the proposed approach over other state-of-the-art RGB-T saliency detection methods.Third,a RGB-T image saliency detection model based on multi-feature collaborative graph learning model is proposed.On the one hand,in previous methods,the graph structure is fixed which only considers the local neighbors,and can't capture more intrinsic relationships between graph nodes.On the other hand,previous methods construct the graph based on the handcraft features and the saliency computation is independent phases.To solve the above problems,first we segment the input RGB and thermal images jointly into a set of superpixels,and extract multi-level deep features from these superpixels for each modality,which are employed as nodes of the graph.Then we collaboratively use hierarchical deep features to jointly learn graph affinity and node saliency in a unified optimization framework.Extensive experiments on all currently RGB-T datasets prove that the results of the proposed collaborative graph learning method perform better against the state-of-the-art RGB-T saliency detection methods.