RGBT Object Tracking Based On Collaborative Correlation Filter

RGBT object tracking based on visible light?RGB?and thermal infrared?T?videos is an emerging hot research topic in the field of computer vision.Through adaptive fusion of visible light and thermal infrared data,RGBT object tracking can effectively deal with the challenge scenarios?such as illumination variation,bad weather and thermal crossover,etc.?that exist in object tracking based on single modality.Therefore,no matter in the field of military defense or civil security,RGBT object tracking has extremely important scientific research value and broad practical application prospects.In recent years,researchers have proposed many RGBT object tracking algorithms based on sparse representation,graph model and deep learning.Although these methods have achieved very good tracking performance,the efficiency of most current RGBT object tracking algorithms cannot achieve the real-time tracking speed.In order to overcome the above problems,this dissertation carry out relevant research on how to introduce visible light and thermal infrared data into the correlation filter tracking framework for RGBT object tracking.On the one hand,it has achieved excellent RGBT tracking accuracy,on the other hand,it has reached the real-time running speed requirements.The main works of this dissertation include the following aspects:First,a collaborative sparse correlation filter model is proposed.In order to suppress the impact of challenging scenarios such as illumination variation,severe weather and background clutter on single modality object tracking algorithms,and considering that most existing RGBT object tracking algorithms cannot take both the tracking performance and efficiency into account at the same time,this dissertation designs a novel collaborative sparse correlation filter model for RGBT object tracking.The proposed approach is able to deploy the intra-and inter-spectral information in the correlation filter tracking framework.Firstly,the proposed algorithm learns the sparse correlation filter for each modality.Due to the advantages inherited from the sparse representation,the learned sparse correlation filters can represent the essential information of the tracked object while being insensitive to noise and errors.Secondly,considering that the motion of the same object in different modalities is synergistic and consistent,we model the interdependence between the sparse correlation filters as a sparse learning problem base on7)_2,1 regularization.Finally,experimental results on GTOT and RGBT210 public benchmark datasets demonstrate the effectiveness and excellence of our proposed algorithm.Second,a soft-consistent correlation filter model is proposed.Due to the heterogeneity between different modalities,it is difficult to fully exploit the complementarity between different modalities if only considering collaboration.In addition,the first work did not perform weighted adaptive fusion of different modalities,which can affect the tracking performance easily.In order to overcome the above problems,this dissertation designs a RGBT object tracking algorithm based on the proposed soft-consistent correlation filter model.Our method takes both the collaboration and the heterogeneity of different spectral information into account for more effective fusion.For the collaboration,we observe that the learned filters should select similar circular shifts such that they have similar motion.While for the heterogeneity,we intend to allow filters have sparse different elements to each other.In addition,we introduce a novel mechanism to fuse the response maps of different modalities for robust visual tracking.Specifically,we calculate the modal weights according to the response maps in the detection phase,and the final response map is obtained by weighted fusion of each modal response map.Finally,experiments on GTOT and RGBT210 public benchmark datasets verify that the proposed algorithm has the comparable tracking accuracy against state-of-the-art RGBT tracking algorithms while meeting the real-time running speed requirements.