Learning Visually Collaborative Models For RGB-T Object Tracking

RGB-T object tracking achieves robust object tracking by combining visible light(RGB)and thermal(T)data,and plays an important role in applications,such as video surveillance,human-computer interaction and intelligent transportation systems.The current RGB-T object tracking methods use a rectangular bounding box as the tracking result of the target,and the rectangular bounding box includes the position and scale information.In some complex environments,the rectangular bounding box of the target could not represent the position and size of the target well.Thus introducing some noisy information.This thesis studies the problem and proposes three visual collaboration models.Aims at constructing robust object feature representation to suppress the background influence in the rectangular bounding box,thereby improving the RGB-T object tracking effect.First,a collaborative manifold ranking model is proposed.In order to obtain accurate object feature representation,by observing the relationship between the object bounding box and the object,we can find that the center of the object bounding box tends to be foreground,the exterior of the bounding box tends to be background,and the object tracking problem can be seen as a task that distinguishes between foreground and background information.The manifold ranking model can distinguish the foreground and the background well.Therefore,this thesis proposes a multimodal tracking algorithm for collaborative manifold ranking model.The target bounding box is first divided into non-overlap patches,the patches are regarded as nodes,and then construct a graph in 8-neighborhood way.The collaborative manifold ranking model is used to learn a weight for each patch to distinguish the foreground and background information,and a weight for each modality to represent their reliability.In order to mitigate the influence of noise on patch weight and modality weight,based on the first ranking result,the collaborative manifold ranking model is used again to obtain the final ranking results.Finally,the patch weigh,the modality weights and the feature of each patch are combined together as the input of the Structured SVM tracking framework for obj ect tracking.Second,a robust cross-modal manifold ranking model is proposed.On the one hand,the imaging mechanisms of different modals are different.In the first model,the heterogeneity between modals is not considered,which has great influence on the tracking performance;on the other hand,the image patch weight is calculated by multiple stages with manifold ranking,which is more time consuming.In order to solve above problems well,based on the traditional manifold ranking model.On the one hand,cross-modal soft consistency constraints is added to achieve better RGB-T information fusion.On the other hand,optimized seed learning stage is added to get reasonable and effective seed.In order to improve the tracking efficiency and get a better closed form solution,the thesis jointly optimizes the image patch weight and reasonable seed in a unified model.The learned image patch weight,combined with the structured support vector machine tracking framework achieves robust object tracking.Third,a local-global collaborative graph representation model is proposed.In the first two models,only the traditional 8 neighborhood method is used for graph construction.The graph construction method only considers local features and does not consider global features.In order to solve above problems,the thesis proposes a tracking method combining global and local information,to suppress the impact of background information.Specifically,on the one hand,the traditional 8 neighbor graph is used to explore the relationship between the local information of the graph.On the other hand,the global low-rank structure between the image patches is used to dynamically learn the global structure of the graph.Combining these two together,the global and local information can be learned.At the same time,the image patch weight is optimized in a semi-supervised manner.Finally,the image patches weight are combined together with the image patch feature as the input of the tracker,which greatly improves the tracking performance.