Research On Target Tracking Method Based On Multi-modal Image Fusion

Visual tracking is a basic task in computer vision,with many real-world application scenarios,including video surveillance,autonomous driving,human-computer interaction,robot control,etc.The task of visual tracking is to predict the size and position of the target in subsequent frames on the premise of knowing the size and position of the target of interest in the initial frame of the video.Although significant progress has been made in target tracking based on monomodal information in the past few decades,the robustness of target tracking remains to be seen when faced with complex challenging scenarios,such as changes in illumination,background clutter,and rainy weather.improve.In order to further improve the performance of target tracking in complex scenes or extreme conditions,more and more researchers are beginning to use the complementary advantages of different modal information for joint tracking,mainly focusing on RGBT tracking that combines visible light and thermal infrared information,and visible light and RGBD tracking with deep information fusion.This paper studies these two types of tracking,and proposes RGBT and RGBD tracking schemes under the framework of related filtering.These two schemes not only achieve excellent tracking accuracy,but also meet the speed requirements of real-time operation.The main research results of this article include the following two aspects:(1)In order to suppress the influence of challenge factors such as rain,haze and illumination changes on the single-mode target tracking method,and at the same time,considering the model drift caused by the model update process,this paper proposes a method based on adaptive regularization and identify the RGBT tracking method of the updated model.Specifically,for the input visible light and thermal infrared images,the search area is determined and the multi-channel feature information is extracted.The extracted feature information and the constructed adaptive spatio-temporal regularization filter model are filtered to obtain each mode.Response graph.In order to make better use of the complementary advantages of the two modes,an adaptive weighted integration scheme is adopted to fuse each modal response graph to determine the location of the target.Finally,in the model update process,a high-confidence discriminant model update method is used Prevent the drift of the model.Extensive experiments have been conducted on the recently published benchmark test sets RGBT210 and RGBT234.The experimental results show that our proposed tracker is superior to the most advanced RGBT tracker in terms of tracking accuracy and speed.(2)In order to robustly cope with challenging factors such as large appearance changes and occlusions during target tracking,this paper proposes an RGBD tracking method based on multi-class feature identification fusion and similar update under the framework of related filtering.Specifically,first use the combination of CNN features and traditional features to perform feature extraction on the input visible light image and depth image,so as to accurately express the target information,and then fuse the extracted features and select the most discriminative feature map In order to remove the influence of redundant noise,secondly,filter with the constructed dynamic regularization filter model to obtain the response map to determine the location of the target,and finally use the adjacent template similar model update to effectively avoid the degradation of the model.A large number of comprehensive experiments on BTB and PTB have been conducted on public data sets.The experimental results show that compared with the latest existing RGBD trackers,the RGBD tracker proposed in this paper has good tracking accuracy and robustness.