Research On Target Tracking Algorithm Based On Deep Learning And Reinforcement Learning

Object tracking is an important problem in computer vision,and has a wide range of applications in video surveillance,intelligent robots,unmanned driving,medical diagnosis and other fields.In recent decades,despite many advances in object tracking algorithms,there are still some challenges that hinder object tracking,such as fast motion,motion blur,background clutter,deformation,illumation changes,out-of-view and so on.Although traditional methods have good computational efficiency and tracking performance,they cannot effectively overcome the above challenges due to the defects in hand-craft features.Since deep learning can bring rich feature representations and reinforcement learning can treat object tracking as a decision-making process to locate object sequentially and learn a good tracking policy,we studied the existing object tracking algorithms,and proposed some improved algorithms in this thesis.The main contents of this thesis are as follows:(1)We designed an improved ACT model,named AACT,based on attention mechanism.In order to improve the classifier in ACT model during online tracking,we introduced the reciprocating training attention mechanism in DAT,and proposed AACT based on the attention mechanism.With the reciprocating training method,the deep classifier was trained to be visual attentive.Attention maps can selectively focus on trmporal-robust features to improve object recognition.Experiments showed that this reciprocating training attention mechanism can effectively improve the tracking success rate of the original ACT algorithm.Also,our AACT had good performance while solving challenges like background clutter,deformation and illumation changes.(2)We proposed an improved Siam RPN model named ACsiam RPN with an inter-frame pre-positioning network learned by a reinforcement learning method Actor-Critic.While Siam RPN has the risk of losing the target out of its small searching area,in order to improve the tracking accuracy and success rate,we proposed ACSiam RPN with 4 times larger searching area,using the inter-frame object motion information to perform the object prepositioning.The pre-positioning network expands the object searching area,and was trained by means of the Actor-Critic method in reinforcement learning.Then,it regresses the object pre-position and modifies the searching area center of Siam RPN,thereby improving the precision rate and success rate of the Siam RPN tracker.Experiments showed that our ACSiam RPN exceeds Siam RPN in terms of precision rate and success rate,running at 65 fps,and still maintain good real-time performance against several advanced object tracking methods.This pre-positioning network trained based on reinforcement learning method can effectively improve the accuracy and success rate of Siam RPN model and have good performance while solving challenges like fast motion,motion blur,illumation changes and scale changes.(3)We proposed an improved Siam RPN model named PWCSiam RPN based on PWCNet optical flow estimation.In order to improve the tracking accuracy and success rate of the Siam RPN model,we used a 4 times larger search strategy and utilized the optical flow estimation network PWC-Net to introduce optical flow information which is helpful to predict the inter-frame object movement trend.Then,we took advantages of Ro IAlign to obtain the target appearance characteristics of the previous frame.Also,GIo U was introduced to form our loss function to train the pre-positioned network that regresses the object pre-position and modifies the searching area center of Siam RPN.Experiments have proved that the tracking accuracy and success rate of PWCSiam RPN exceed that of Siam RPN,and the running speed is 20 fps,which is close to real-time performance.Also,our PWCSiam RPN had good performance while solving challenges like fast motion,deformation,out-of-view and scale changes.Our methods still have certain advantages against several advanced object tracking methods.In addition,the comparative experiments of various improved Siam RPN models and variants proposed in this thesis prove that expanding the search area and introducing interframe motion information and optical flow information for inter-frame pre-positioning before Siam RPN tracking can effectively improve the tracking effect of the Siam RPN model.