Research On Key Technologies Of Abnormal Behavior Analysis Based On Trajectories

Abnormal behavior detection in video data has wide applications in traffic monitoring and social security.In recent years,it has been one of the hot research topics in computer vision area.In this paper,we focus on the information extraction and anomaly analysis on the behaviors of moving targets in surveillance video.The trajectories of moving targets can provide global information for target motion patterns.Compared with raw video data,trajectories are compact for storage,transmission and analysis,thus especially suitable for long-term traffic video surveillance on vehicles.Therefore,we utilize the trajectories of moving targets as the representation of target behavior and study several key problems in abnormal behavior detection.On the one hand,in order to obtain the motion information of key target objects in long-term surveillance,we study the visual object tracking methods for extracting trajectories of single targets in surveillance videos and study the online tracking method for the trajectory extraction of multiple objects in complex scenes.On the other hand,we address the problem of anomaly detection based on target trajectories.To detect the abnormal trajectories that do not conform to the mainstream pattern,we propose methods by the discriminative analysis based on distance metrics and by the generative analysis based on motion regularity learning.The contributions of this thesis are four fold and can be summarized as follows:(1)We propose a method that extracts the trajectory for a single target based on a newsaliency prior context model.The tracking procedure are prone to drift in some caseswhere targets are deformable or move in non-linear manners.In order to improvetracking robustness,we use both the spectral visual saliency analysis and low-levelappearance features to distinguish the foreground target from the context backgroundregion and construct a saliency prior context model.For a better accuracy,thelocation of target is estimated based on both this model and the spatial context modelaccumulated across time.In addition,the computation in model construction andtracking iteration is accelerated with fast discrete spectral transform to increase thetracking efficiency.Experimental results show that visual saliency can assist locatingthe target in the context region and improve the tracking robustness.Compared withthe existing methods,our algorithm improves the accuracy in extracting targettrajectories and achieves a good balance between speed and accuracy.(2)We propose an online method for extracting multiple target trajectories using theconstraints based on the motion vectors and refined appearance features.The task ofonline multi-object tracking needs to associate the targets in different video frames.However,it is challenging due to false alarms in detection and the ambiguiltybetween multiple objects.In order to tackle these problems,we obtain motionconstraint based on the optical flow vectors and refine the target appearance basedon visual saliency methods.We propose a new cost function constructed bycombining the motion constraint,appearance constraint,spatial constraint and sizeconstraint.Thus,we obtain the association matrix and implement the online multi-object tracking under the tracking-by-detection framework.Experiments on publicdatasets show that our method is able to effectively eliminate the effects caused byfalse detection.Compared with the state-of-the-art algorithms,our method achievesfavorable results and shows a significantly low false alarm rate.(3)We propose a new trajectory distance metric based on a new autoencoder structureand analyze abnormal behaviors based on trajectory distances.For the targettrajectory data obtained from surveillance videos,to discriminate various abnormaltrajecotries from normal ones,we use the recurrent neural networks to capture thedynamic temporal characteristics of the sequence data.For two trajectories,we usethe autoencoder based on recurrent neural networks to learn both trajectory modelsand calculate the cross reconstruction error to obtain the distance between the twosamples.Then,we use the nearest neighbor detector with an adaptive threshold toevaluate the abnormality of trajectories.Comprehensive experiments show that theproposed trajectory distance metric can effectively distinguish various abnormaltrajectory motion patterns and detect abnormal trajectories under various complexscenarios.(4)We propose a method to detect abnormal trajectories based on a sequence-to-sequence regularity model.In order to deal with the complexity of real-worldtrajectories,we construct a sequence-to-sequence autoencoder model based onrecurrent neural networks and propose a new loss function based on trajectorylocation,speed and orientation for model optimization.As such,we use the model tolearn the normal pattern from trajectory data and reconstruct the test samples withthe learned model.The anomaly score is computed with the error of reconstructingthe sample with the learned model.Experiments on real-world datasets show that themethod proposed in this thesis can learn the mainstream motion patterns fromunlabeled trajectory data that mixed with a small amount of abnormal samples.Compared with the existing methods,the proposed method performs significantlybetter in detection rate.In summary,this thesis focuses on the several key technologies of abnormal behavior detection based on moving target trajectories and studies on both target trajectory extraction and abnormal behavior analysis.Corresponding methods are proposed to form an entire solution for detecting abnormal behaviors in constrained surveillance videos.This solution is especially suitable for analyzing vehicle behaviors in traffic monitoring.