Research On RGB-T Object Tracking Algorithm Based On Deep Networ

Object tracking is an important and fundamental problem in the field of computer vision,and it is also one of the hotspots and difficulties in recent years.Object tracking has been widely used in the fields of intelligent driving,intelligent monitoring,and UAV aerial photography.RGB cameras can capture the color and texture information of objects,but the image quality is easily influenced by lighting conditions.Thermal infrared cameras capture the thermal radiation emitted by the object itself,eliminating the interference of factors such as light and weather but lacking the corresponding information of color and texture.What’s more,thermal infrared images are susceptible to thermal crossover.RGB-T(RGB and Thermal infrared)tracking is a method to track objects by using fused visible modality and thermal infrared modality information,which has attracted much attention in recent years.How to mine the complementary information between RGB images and thermal infrared images,and improve the robustness of object tracking through RGB-thermal infrared fusion,is an important issue in the current RGB-T target tracking task.In recent years,methods based on deep learning have dominated the RGB-T object tracking field.This paper introduces the researches in recent years,and conducts in-depth research on the RGB-T object tracking method based on deep network to make full use of the powerful representation ability of deep neural network.The main work of this paper are as follows:(1)A high speed RGB-T object tracking algorithm based on deep dual-siamese network is proposed.Tracking is modeled as a similarity measurement task using a two-stream siamese network architecture,a channel attention mechanism is used to propagate information between two modalities,and a spatial attention mechanism is used to suppress background noises.A dual-modality region proposal network is constructed to classify the target location and regress the appearance,and use response-level fusion to improve the tracking speed.Finally,an adaptive peak selection module is used to select the optimal candidate proposal.The experimental results show that the method can perform RGB-T tracking with high speed and robustness.(2)A challenge-aware RGB-T object tracking algorithm is proposed.For the same video sequence,the different branches of the challenge-driven branches are used for feature extraction,and the features of RGB and Tir(Thermal-infrared)modalities are fused at this stage.The fusion features output by each challenge branch are sent to the challenge-aware feature aggregation network to calculate the weight under each challenge and perform weighted aggregation.The aggregated features are used to calculate the overall classification response map and regression response map through the region proposal network.Finally,the distractor-suppress proposal selection module is used to suppress the similary distractor in the background,and the optimal proposal is selected.The experimental results show that our algorithm achieves a higher level of tracking accuracy than other state-of-the-art methods while satisfying the real-time tracking performance.(3)A deep network-based visible-thermal infrared object tracking system is implemented.The system includes four main modules: visible-thermal infrared data enhancement module,neural network model training framework,visible-thermal infrared video sequence tracking module,and visible-thermal infrared tracking evaluation module.The system is developed by using Py Qt5,integrates the two algorithms proposed in this paper and other mainstream methods,and can satisfy funcations of data enhancement,model training,tracking testing,and tracking evaluation through a graphical interface.