Online Multi-object Tracking In Cluttered Scenes

Multi-object tracking aims to estimate the trajectories of multiple objects in a video,which acts as an input for the further analyze and understanding of the video content.It is a highly active problem in the domain of computer vision and pattern recognition,and is of great importance for various applications such as video surveillance,visual navigation,intelligent transportation system,sports training,etc.Although much progress has been reported,multi-object tracking is still a very challenging problem in cluttered scenes,due to the presence of large amount of objects,similar-looking appearances,background clutters,and frequent occlusions.In this thesis,we adopt a tracking-by-detection framework to divide the task of multi-object tracking into object detection and data association,and focus on how to design effective algorithms to track multiple objects online in cluttered scenes.A novel appearance modeling approach,called temporal dynamic appearance modeling,is proposed to exploit the temporal dynamic characteristics within temporal appearance sequences to discriminate different persons.The resulting appearance model provides accurate appearance affinities to guide data association,and thus improves the performance of multi-person tracking.The Hidden Markov Model(HMM)is employed to capture both the temporal dynamic and the spatial structure when the appearances of persons change over time,where the temporal dynamic of appearance changes is indicated by the underlying Markov process,and the spatial structure is captured by multiple observation densities.The appearance model is learned incrementally based on an online Expectation-Maximization(EM)algorithm.To facilitate temporal dynamic appearance modeling,a feature selection algorithm is presented to describe the appearance variations with mid-level semantic features.Reliable tracking of multiple persons in complex scenes is achieved by incorporating the learned model into an online tracking-by-detection framework.The experiments on the MOTChallenge 2015 test bench show that our method is able to preserve identity when frequent and close interactions between similar-looking persons occur.To alleviate the large dependency between the performance of multi-object tracking and the performance of the employed object detector,a novel framework is presented to jointly optimize the detections and trajectories,where a sequential trajectory prior is incorporated to guide the optimization.Under the Bayesian probabilistic framework,the object detection and data association problems are formulated as two collaborative Maximum a Posteriori(MAP)estimation stages.The sequential trajectory prior is extracted from the previous information contained in the existing trajectories,and is incorporated into the two MAP stages.It simultaneously guides the exploration of optimal detections and enhances the association correctness between trajectories and detections.By solving the MAP estimations of object detection and data association,the accurate trajectories of multiple objects are built sequentially even when noisy detection responses and frequent inter-object interactions occur.The experiments demonstrate that our method provides superior tracking performance in various cluttered scenes such as campus,street,parking lot,etc.To further improve the performance of online multi-object tracking in complex scenes,a novel hybrid local/global data association framework is presented to extent the local data association between adjacent frames to account for more hypotheses from multiple frames,which characterizes the superiorities of both local and global data association methods.A min-cost multi-commodity network flow formulation is exploited to find optimal data associations of multiple existing trajectories over multiple frames.Detections are considered as vertices of the network and the connections between detections are modeled as edges.Each existing trajectory is supposed to be a specific commodity,and its optimal associations can be found by sending specific commodity flows through the network.The flow costs of the edges in the network are extended for multiple commodities to account for the target-specific information contained in the existing trajectories.A dummy commodity corresponding to a target-independent model is added to automatically identifies new objects.To ensure the efficiency of online tracking,an efficient near-optimal solution to the min-cost multi-commodity flow problem is presented with empirical certificates of its sub-optimality.The comprehensive experiments on real data demonstrate the superior tracking performance of our method in various challenging situations especially when the objects suffer from long-term occlusions and abrupt motions.