Research On Object Tracking Algorithms Based On Asymmetric Discriminant Correlation Filters

As an important research topic in the field of computer vision,object tracking has broad development and application prospects in many fields,such as intelligent monitoring,intel-ligent transportation,biomedicine,agriculture and military applications.In recent years,the technology of object tracking has made great progress,and the application of deep learning in this field has developed rapidly.However,object tracking is still facing a series of challenges,such as large object deformation,large visual angle change,occlusion,various scale changes,fast object motion,low resolution,real-time requirements and so on.Especially in the field of UAV(Unmanned Aerial Vehicle)tracking,due to the limitations of onboard computing re-sources,battery capacity and maximum load of UAV and the requirement of low power,the deployment of deep learning-based tracking algorithms in UAV is still not feasible.The dis-criminative correlation filters(DCF)-based tracking algorithms have high CPU speed and good tracking accuracy,which though are not as good as the tracking algorithms based on end-to-end deep learning methods,they still attract a lot of attention of researchers and engineers,espe-cially in the field of UAV tracking.Therefore,the DCF-based tracking algorithms still have important research value and application prospects.Although DCF-based tracking algorithms have greatly improved compared with traditional obj ect tracking algorithms,there remain problems to be further studied,for example,the transla-tional equivariance of correlation(convolution)operator can not faithfully reflect the translation of the target,its symmetry will reduce the scalability of the model and the wildly used discrim-inative scale estimation in DCF-based algorithms is inaccurate.To deal with these problems,this Ph.D.thesis proposes the concepts of generalized translational equivariance and weak gen-eralized translational equivariance,generalizes the correlation(convolution)operator,proposes an asymmetric discriminative correlation filter based on asymmetric correlation(convolution)operator and as well an optimization algorithm along with an improved one to solve it.In ad-dition,the discriminative scale estimation is improved by the image segmentation algorithm GrabCut.The innovations and contributions in this thesis are as follows:1.That the discriminative correlation filter constructed by correlation operator is equiva-lent to the discriminative convolution filter constructed by convolution operator is proved,and a concept of generalized translational equivariance and an asymmetric discriminative correlation filter are proposed.The tracking algorithm based on discriminative correlation filter considers visual tracking as a problem of matching the feature template of the target against the candidate regions of the detection sample in which the correlation filter is the means to calculate the sim-ilarity.Despite that convolution is usually thought of an action of an observation system on an input signal,it can be used to achieve the purpose of tracking as well.However,the relation?ship between them has not been strictly discussed in previous literature.This thesis proves that the discriminative correlation filter and the discriminative convolution filter are equivalent in the sense of equal least mean square error of estimation under the condition that the ideal filter response is a Gaussian function and the optimal solutions exist.The translational equivariance of the convolution(correlation)operator in DCF guarantees that the filter response faithfully reflect the target's translation if the detected sample just represents the target itself.However,cyclic convolution(correlation)is a symmetric operator,that is,two operants of finite discrete signals expand with the same period and the result of convolution result remains unchanged whether exchanging the two signals or not,which brings in some problems that can not be ig-nored in tracking applications.On the one hand,this symmetry requires the size of the filter and that of the sample to be equal,which reduces the scalability of the model because the number of filter parameters and the complexity of the model grow nonlinearly with the sample size.On the other hand,the translational equivariance of the symmetric convolution(correlation)is defined by the whole sample.If the ratio of the sample size over the target size is greater than 1,the sample contains backgrounds,when the response will not reflect the translation of the target but the mixed effect of the target and the background instead.This problem,however,has not been recognized and even ignored completely in the field of object tracking.In order to clarify this problem,we need to define a new translational equivariance,namely a generalized transla-tional equivariance,which describes that the translation of convolution result is with respect to the translation of the target only rather than the whole sample,which is the right translational equivariance required in object tracking.However,it is difficult to find an operator that has this property,so we define the concept of weak generalized translational equivalence.Meanwhile,we propose an asymmetric convolution operator and prove that the operator satisfies the weak generalized translation equivalence under certain conditions.The filter based on this asymmet-ric convolution operator is called an asymmetric discriminative correlation filter(ADCF).We prove that the coefficient matrix of the normal equation derived from ADCF is a block ma-trix,and each block is a two-level block Toeplitz matrix,which generalizes the case that each block is a circulant matrix in DCF.With this,we design a fast algorithm to compute product of the block Toeplitz matrix and a vector for solving ADCF.Compared with DCF-based tracking algorithms,the tracking accuracy and precision of ADCF are significantly improved in cases where the generalized translation equivalence becomes important.2.A residue-aware asymmetric discriminative correlation filter is proposed.Although the BACF(background-aware correlation filter)and our ADCF are different in starting point and mathematical form,they are essentially equivalent but optimized differently.We construct an approximate algorithm for solving ADCF based on the structure of two-level block Toeplitz matrices of its normal equation,wile BACF decomposes its optimization problem into indepen-dent subproblems using ADMM.The efficiency of parallel algorithm is much higher than the method we proposed.However,using ADMM to solve the asymmetric discriminative correla-tion filter still faces the problems of slow convergence and numerical instability.In particular,the tracking precision and accuracy of ARCF-HC which is based on BACF surpasses all pre-vious DCF-based trackers in UAV tracking,but its efficiency is low,requiring five ADMM iterations per frame,which is hard to meet the real-time requirements of UAV tracking.In-spired by residual representation and residual learning,especially that the deep residuals net-work ResNets can reduce shattered gradients,improve the network learning and convergence speed and improve the numerical stability,we propose a residue-arware asymmetric discrimina-tive correlation filter based on the residual nature of adjacent video frames,which significantly improves the convergence speed and stability of the optimization process,which only needs two ADMM iterations per frame,saving a lot of time compared with ARCF-HC.Moreover,the residue-aware asymmetric discriminative correlation filter can be considered as compounding several objective functions and also shows advantages over ARCF-HC in precision and accu-racy.In addition,we also succeed to add spatial and temporal regularizations to improve the performance of the residue-aware asymmetric discriminative correlation filter with only little additional time cost,because just some extra addition operations are involved.3.The method to refine the discriminative scale estimation using the segmentation algo-rithm GrabCut is proposed.Most DCF-based trackers use the discriminative scale estimation to estimate the scale change of targets since it was proposed.And there is only few work attempts to improve the discriminative scale estimation for object tracking.However,scale estimation directly affects the precision and accuracy of tracking algorithms.On the one hand,the eval-uation of the precision and accuracy of tracking algorithms directly depends on the estimated scale.On the other hand,the filter update is based on the scale estimation too.If the estimated scale is larger than the actual scale,the sample representing the target will contain backgrounds,whereas if the estimated scale is smaller than the actual scale,it represents only part of the tar-get.As time goes on,the error of scale estimation will accumulate,which may eventually lead to the tracking failure.Moreover,since the size of the filter is much smaller than that of the detected sample ADCF is more sensitive to scale errors,therefore it needs more accurate scale estimation.The method to refine the discriminative scale estimation using the segmentation algorithm GrabCut is proposed in this thesis,which has improved the precision and accuracy of discriminative correlation filter tracking algorithm in UAV target tracking as shown by exten-sive experiments.In particular,our method is so generic that it can be easily incorporated into any existing tracking algorithm based on discriminative scale estimation.