Online Multi-Object Tracking In Video Based On Interframe Continuity

As one of the most important computer vision tasks,Multi-Object Tracking(MOT)in video aims at estimating the trajectories of all interested objects in the video.The trajectories produced by MOT provide technical supports for video understanding and scene perception,resulting in the various applications of MOT,such as auto-dri ving,intelligent surveillance,drones,medical imaging,and so on.Early works of MOT process the video in an offline manner.The frames in a time window are processed simultaneously,introducing non-negligible latency.In recent years,researchers have pay more attention to online MOT methods to meet the requirement of low latency in real applications.Different from offline methods,online methods process videos frame-by-frame,which means that the online methods possess a much lower latency and a much broader range of applications.Although online MOT methods have achieved success to some extent in many applications,there exist some challenges when applied to real applications:existing methods restore the frames in a video back to RGB images and all images need to be processed samely and costly,resulting in a slow tracking speed;existing methods only utilize the individual feature of each object for tracking,leaving the relationships between different objects unexplored.The tracking ability is unstable in complex scenes,where occlusions between different objects happen and different objects share similar appearances;existing methods need the identity annotation for the training of appearance module.The appearance feature can be used to re-identify the missed objects after their reappearance.However,the positions of highly occluded objects still can not be estimated when they are missed by the detector.In addition,the identity annotation is expensive.To handle these challenges,this dissertation studies on online multi-object tracking.The main works and innovations are as follows:1.Propose an online MOT method in compressed domainThe frames in compressed video can be divided into key and non-key frames.Among them,the key frames contain all the information for decompression,while the non-key frames are encoded with residuals and motion vectors.To achieve a faster tracking speed,different tracking strategies are designed for key and nonkey frames.For the key frames,a position sensitive appearance module is designed,which computes the similarity between different appearance features effectively.What’s more,the appearance module can be trained and optimized with the object detector jointly.For the non-key frames,a light tracking network is designed to predict the bounding boxes of objects based on motion vectors and residuals.Compared with key frames,object detection is not performed and the appearance/disappearance of objects are not considered for non-key frames,which reduces the computional cost.Experimental results demonstrate that the tracking speed can be greatly boosted at the cost of negligible negative impact on tracking performance.2.Propose an online MOT method based on graph similarity modelThe interframe continuity of video makes the relationships between different objects in adjacent frames almost the same.To improve the robustness of the tracking methods by utilizing the relationships between different objects,a graph similarity model is designed.Specifically,a directed graph is constructed for each object based on the appearance features of objects and the relative positions between different objects.Through graph matching,the similarity between different objects can be calculated.In addition,with the assumption that the topologies of graphs in adjacent frames are invariant,the position of missed objects can be estimated.Compared with existing methods,the proposed method is more robust thanks to the consideration of both appearance features and relative position features.Experimental results show that the tracking performance of existing methods can be improved with the help of the proposed graph similarity model.3.Propose an online MOT method with unsupervised re-identification and occlusion estimationTo reduce the reliance of tracking methods on data annotations,the proposed method learns the re-identification module in an unsupervised way.Specifically,given the union of objects in adjacent frames,the similarity score between any two objects can be obtained.Through the application of intra-frame and inter-frame constraints on these similarity scores,the re-identification module is learned in an unsupervised way.However,the re-identification module can only be used to reidentify the missed objects after their reappearance,leaving the missed positions unestimated.To handle this,an occlusion estimation module is further designed.When an object is missed,its position can be well estimated using the detected occlusions and its motion information,improving the occlusion handling cabability of the proposed method.Experimental results show that the unsupervised re-identification module is comparable to those supervised counterparts in existing methods,and the occlusion estimation module can be used to improve the tracking performance of existing methods greatly.4.Design an online MOT demonstration systemThis demonstration system combines the above three tracking methods effectively to obtain a real-time and robust tracking method,which also has low dependence on data annotations.In addition,there exist a surveillance system and an interactive online multi-object tracking system in the desinged demonstration system,among which,the former one supports multi-camera and multi-object tracking while the latter one allows users to process and analysis different videos conveniently.