Research On Multi-modal Object Tracking Based On Structural Sparse Representation Model

With the widespread popularity of intelligent video surveillance systems,the advancement of computer vision theory has been promoted.Object tracking has been attached more and more attention in both theoretical research and practical application,and it has become an important topic in the field of computer vision.Due to imaging limitations of single kind of sensors,object tracking only using visual spectrum information still faces many challenges,such as background clutter,object deformation and illumination changes.To overcome this obstacle,multi-modal integration is applied in visual tracking,and has achieved great improvement over single source.In recent years,multi-modal object tracking methods,especially visible-infrared object tracking,mainly focus on the sparse representation appearance models due to their capability of suppressing noise and errors.However,most of those methods only consider holistic or intrinsic structure among and inside target candidates and ignore adaptively exploiting grayscale and thermal information based on their reliabilities.In this dissertation,we study two major problems of multi-modal object tracking based on sparse representation models.On the one hand,how to combine the two modal videos while taking into account the correlation between the sparse representation coefficients in each modality to achieve more robust multi-modal object tracking.On the other hand,how to construct a target sparse representation model can systematically represent the global and local appearance of the target while suppressing the influence of the background information.The major works are summarized as follows:(1)To cope with different scene conditions and appropriately select different modal information to achieve a more accurate expression of the target appearance,a multi-modal object tracking method is proposed in the particle filter framework.The proposed method relies on a collaborative sparse representation model,which introduces modal weights and updates them dynamically over time.Then,in order to ensure consistency of sparse reconstruction coefficients between different modalities,the cross-mode consistency constraint of sparse reconstruction matrix is introduced in this model.Finally,we conduct plenty of experiments to verify the effectiveness of the proposed approach in the public tracking dataset.(2)In order to express the object appearance better and cope with the large appearance changes during target tracking,we proposed a structural sparse representation approach for multi-modal object tracking under the particle filter framework.The proposed structural sparse representation model can preserve object local and global information between the candidate particles and the target template set by taking different scales of patches into account.Moreover,we observe that different local patches within the object region have different contributions to the tracking results,and thus integrate a weight for each patch into the proposed model to achieve robust representations.After solving the model jointly,the candidate sample with the largest likelihood is selected as the tracking results.Finally,a large number of verification and comparative experiments are carried out on public tracking dataset and verify the effectiveness of the proposed approach.