Detection Based Data Association Method For Multiple Object Tracking

Multiple object tracking (MOT) in video sequence is an important task in the field of computer vision. It has been widely applied in such areas as national defense, visual surveil-lance, intelligent navigation/aided driving, intelligent robot, human behavior analysis, video retrieval, and bio-medicine. The aim of MOT is to reconstruct each target's moving trajec-tory in video sequence. However, multiple object tracking remains a challenging problem due to poor image quality, noise and clutter, variation of appearance and moving pattern, unknown number of tracked targets, occlusions, etc. Lots of theoretical and technical issues are still to be resolved.In recent ten years, tracking-by-detection approaches have become increasingly pop-ular thanks to the significant improvements in object detection techniques. As a current mainstream method, detection responses generated by a detector are taken as input, and those responses belonging to the same IDs are associated to obtain the trajectory for each target. For detection based data association, one key issue lies in design of the affinity mod-el. An appropriate affinity model should contain observations that correspond to the inherent attribute of targets, and provide reliable associations in noisy and complicated scenario.According to the advancing front in the field, this dissertation focuses on deep investi-gation to affinity model and its application in multi-target tracking. The main contributions of this thesis are given below.(1) Multiple object tracking based on Hough Forest learningDue to the competitiveness of hierarchical approach, this thesis proposes a Hough For-est (HF) classifier based algorithm for multiple object tracking, under the hierarchical as-sociation framework. Short but reliable tracklets are firstly generated by a conservative associating strategy. For each level, discriminative features in terms of appearance and motion are then extracted from the set of tracklets, and training samples are generated to construct Hough Forest. In test stage, statistics stored in the leaf nodes of the HF are uti-lized to estimate the linking probability between tracklets, and the association problem is formulated as a Maximum-A-Posteriori (MAP) estimation. The effectiveness of the Hough Forest based affinity model is demonstrated by experimental results. Compared with several state-of-the-art methods, the proposed method has achieved competitive performance.(2) Matching isolated responses based on occlusion reasoningOwing to the complex scene and frequent occlusions, even the state-of-the-art object detectors can not work perfectly. Common errors include missed detections, false alarms, and imprecise localization. Furthermore, some missed detections are also resulted from the conservative association which is adopted to generate reliable tracklets. The issues above will result in isolated responses that cannot be linked with any trajectory in the final tracking output, and thus lead to the increase in trajectory gaps and decrease in trajectory consistency. To address this issue, a novel occlusion reasoning model is presented to infer the occluded and non-occluded region for occluded targets. Based on occlusion reasoning, we design fusion feature for each isolated response, and introduce a matching scheme between an isolated detection and a particular target. In this way, each isolated detection is assigned a corresponding trajectory. It is worth noting that as a post-processing step to fill trajectory gaps, the method in this chapter can be applied in those tracking algorithms which adopt tracklets association strategy.(3) Multi-target tracking based on Hough Forest Conditional Random FieldMore recently, conditional random field (CRF) has been received much attention in multi-target tracking field. However, as the core of CRF based approach, the parameter learning and inference of CRF models are often intractable. Traditionally, approximate or heuristic algorithms are used to parameter estimation, and the obtained inference solution are often locally optimal. To tackle this problem, Hough Forest Conditional Random Field (HFRF) model is proposed in which CRF inference is realized by acceptance rate of MH jumps that is computed via SW-cuts algorithm, and the probabilities required in CRF infer-ence are estimated by Hough Forest. By integrating the CRF learning and inference into a unified computational framework, HFRF bypasses the intractable parameters learning in the classical CRF based tracking methods. Furthermore, different from traditional CRF model, we additionally introduce a binary hidden variable for each edge in HFCRF model, which extends two-tuples data structure to triple case. Then more relative interactions in the spatio-temporal domain can be considered to help optimize algorithm and improve tracking performance.(4) Multiple object tracking based on data joint formulationGenerally, traditional association models are constructed according to the data' s difference, such as computing distance between two features. This process is in fact equivalent to the dimension reduction, leading to loss of the separability that exists in the original data. To address this issue, a multi-target tracking algorithm is presented, in which affinity probabilities are modeled by data joint formulation under the classical CRF framework. Specifically, we model pairwise potential functions to describe the correlation between tracklets, and define higher order label cost (i.e. regular term) to constrain the number of targets in the solution. The association cost is set up as an energy function, on the basis of pairwise potential and regular term. Finally, the labeling task in CRF model is then formulated as cost minimization problem. It is noted that the pairwise potential functions are molded by data joint formulation under the two hypotheses, upon which the probabilities are estimated in terms of compatibility and repellency to conduct the CRF inference. Another trait of the method lies in that by utilizing the statistics about sample labels distribution stored in the leaf nodes of Hough forest, the probabilities under the two hypotheses can be estimated in a nonparametric manner. Experiments conducted on several public datasets demonstrate the effectiveness of the proposed method. The compatibility and repellency about data association model provides a new pathway for multiple object tracking.