Deep Graph Matching Based Data Association For Multiple Object Tracking

As a fundamental research in computer vision field,Multi-object tracking(MOT)has a range of applications such as automatic driving,video surveillance,action recognition,etc.Currently,a popular paradigm in MOT community is the detection-based data association approach,where the regions of targets are located in frame by frame by a detector pre-trained off-line and then an association algorithm is utilized to identify each target.Following the tracking-by-detection framework,this paper treats multi-object tracking problem as a bipartite graph matching between the tracked trajectories and detection responses in current frame.The core of the research lies in constructing association cost and inference for object state.The main contents of the paper are as follows:(1)A linear programming based MOT method is realized by posing bipartite graph matching as a linear programming problem.Particularly,to obtain a better association cost,metric learning is incorporated into neural network based feature representation.A siamese network with contrastive loss and an appearance module with triple loss are designed,respectively.The Hungarian algorithm is used to solve linear programming which optimizes the inference.Experiments have verified the advantage brought by the proposed metric learning based feature modelling,as well as the feasibility of multi-object tracking with bipartite graph matching.(2)A MOT method is proposed based on deep graph matching.To mitigate the interference by occlusions and similar targets,a multiple-cue feature representation learning is design using appearance and motion information.To address the issue that linear programming is unable to model higher order dependencies among targets,a bipartite graph optimization is developed based on deep graph matching,where spatial-temporal constraints among targets are modeled by defining the affinity matrix with second order.Finally,with a realizable form of neural network,the graph state inference is achieved using power iterations.The proposed model has the following advantages.First,the multiple-cue feature learning and fusing strategy is capable of alleviating the association feilure caused by objects' mutual occlusions.Second,feature representation learning and graph matching inference are optimized jointly in an end-to-end manner.lastly,the original linear programming based association is extended to a quadratic programming problem,which better exploits the correlations among targets.The experimental results demonstrate the validity of the presented model.Compared with several state-of-the-art approaches,the proposed algorithm achieves comparable or superior tracking performance.(3)The above deep graph matching mathod has a higher computation complexity due to construction of the second-order affinity matrix.To overcome this limitation,the paper proposes a deep graph matching MOT algorithm which is based on feature representation learning with graph embedding.By means of the graph embedding technique,topological information of nodes is encoded into feature modeling.Thus feature message between neighboring nodes is propagated,reducing the computation complexity of the affinity matrix.Comparative experiments have shown that the running speed is accelerated with no loss of tracking accuracy.On the public MOT16 data set,the presented method performes favorably against several state-of-the-art approaches.