Researches On Tracking By Detection Based Visual Multi-object Tracking

The main task of visual multi-object tracking is to locate and recognize multiple different objects in each frame of the video sequences and to restore all objects' trajectories in the video.Visual multi-object tracking is one of the basic issues in computer vision,which provides important supports for many high-level applications of computer vision.In recent years,the convolution neural network,as the representative of the deep learning,has achieved a great success.Among them,the object detection that realizes the location and recognition makes a significant breakthrough,tracking by detection framework thus drives the multi-object tracking algorithm to more complex and real world scenes.However,in complex cases where the number of objects is large and the objects are frequently occluded to each other,even the best object detector fails to detect all objects exactly,there are many missed and false alarms in the video frames.Hence,how to effectively associate the detection response with noisy from different frames to restore the complete trajectory of each object is the main challenge facing by tracking by detection method.This paper focuses on the tracking by detection based multi-object tracking algorithm.To improve the performance of multi-target tracking in complex visual scene,this paper studies the appearance modeling,motion modeling,occlusion handling and the data association issues carefully,and explores how to effectively deal with these problems in online,offline and semi-online manner when designing multi-object tracking algorithms.In order to furtherly improve the robustness of multi-target tracking,this paper also investigates the case of multi-target tracking with multiple cameras.In particular,in the online multi-target tracking scenario,an online multi-target tracking model based on quadratic pseudo Boolean optimization is proposed to solve the phenomenon of "hijacking" that results from object occlusions.In the offline multi-target tracking scenario,a offline multi-target tracking model based on interaction analyses is proposed to deal with object occlusions and the appearance and motion consistency of each object.Then,a semi-online multi-object tracking model based on Markov random field is proposed in order to simultaneously explore the universality of online model and the robustness of offline model.Finally,to overcome the limited information under single camera,a multiple cameras multi-object tracking model based on object re-identification is proposed to studies the information sharing strategy from multiple cameras.The innovation of this paper mainly includes:1.online multi-object tracking by quadratic pseudo-Boolean optimizationIn this paper,we focus on the online multi-target tracking problem in complex scenes.To solve the problem of trajectory recovery after short-term collisions,we study the data association model with the occlusion handling.In particular,this paper proposes a quadratic pseudo-Boolean optimization data association model.For any object pair that may occlude,the model designs an corresponding quadratic term in the Boolean objective to distinguish the pair with more refined appearance model.At the same time,in order to make the trackers have a better understanding of the changing feature of objects,this paper designs a online updating appearance model for each individual object.2.Interacting tracklets for offline multi-object trackingIn this paper,we investigate the close interaction and distant interaction of tracklets,and design the tracklets interaction analysis model for multi-target tracking algorithm.To be specific,the close interaction deals with occlusions when they come to close,to distinguish these close objects in the crowded scene.The distant interaction,however,deals with the association among three tracklets,by holding the consistency of appearance and motion of each individual,to improve the tracking performance.Meanwhile,in order to speed up the optimization problem of the designed model,this paper transfers the optimization to a graph cut based energy minimization,and theoretically analyzes the advantages of energy minimization to solve the current model.3.semi-online multi-object tracking based on Markov random fieldTacking into account of a small amount of frame from the future,on the one hand,do not seriously weaken the tracking speed,while on the other hand,can significantly improve the stability of the trackers.In this paper,a semi-online multi-object tracking method based on the Markov random field model is proposed,which evaluate the temporal and spatial similarity between the tracker and the detection response and also the similarity between different detection responses to improve the tracking performances.At last,this paper introduces a label cost item in the Markov random field to estimates the changing number of objects.4.multiple cameras multi-object tracking by re-identificationMulti-target tracking algorithm with multiple cameras exploits the relationship of spacetime-view between detection responses to associate them,so as to track different objects.This paper introduces object re-identification techniques to match the objects in different fields.By sharing the detection response information from multiple cameras,the model proposed can significantly improve the performance of multi-target tracking.This paper also proposes a local ?-expansion method to speed up the multiple camera tracker.