Research And Implementation Of Action Recognition Method In Surveillance Scenarios

With the extensive research and application of static image recognition using deep learning,researches of video understanding with deep learning has gradually become an important direction of computer vision researches.Intelligent city surveillance is one of the most popular one among all video understanding topics.Since surveillance videos have larger view and multiple activity instances may happen simultaneously in different part of the video,action recognition in surveillance scenarios not only recognizes the class of action,but also locate the action instance spatially and temporally.This thesis introduces a video action recognition framework consists of object recognition,multi-object tracking,action recognition and temporal localization.Firstly,we use pre-trained YOLO v3 model to detect the human or vehicle objects and their spatial location in the videos.And we use Kalman Filter and Hungarian algorithm to track movement of all the targets in videos.The action recognition candidate sequences are obtained according to the movement trajectory of all targets.Secondly,as for action recognition and temporal localization,we propose the Pseudo-Two-Stream action recognition network consists of mixed convolution residual network and representation flow network.The mixed convolution residual network,constructed with 2D conv and 3-view-deconstructed conv layer,is capable of extracting spatial-temporal feature with fewer parameters and lower computation cost.The representation flow network further captures the motion feature by extracting representation flow from adjacent frames of video.The fusion of the results from both network improves the recognition accuracy.UCF-101 dataset is used to evaluate the accuracy of action recognition part.Our method without representation flow network slightly improve the accuracy with less parameters,and the Pseudo-Two-Stream network with representation flow network effectively boost the accuracy with high recognition speed.As for the evaluation of the action recognition framework,we run the test on VIRAT dataset from ActEV.Our framework achieved lower miss rate comparing to the result we acquired in ActEV18.