Continuous Action Recognition Method Based On Deep Learning

Human action recognition is one of the hot research topics in computer vision,pattern recognition and other related fields,which has broad application prospects in many fields,like video surveillance,intelligent security,and athlete-assisted training,etc.In the past,the methods of human action recognition relied heavily on clipped videos(strongly supervised single-action recognition).Since the amount of data increase sharply everyday and lots of manpower is needed to obtain large number of video clips which are cut accurately and neatly,it's difficult to meet people's demands on applications which need massive data.How to recognize continuous human actions directly with the videos that are not accurately divided(weakly supervised continuous multi-action recognition)has become an important topic for the scholars at home and abroad.Under the rapid development of artificial intelligence,deep learning based method has made great progress and achieved good results in the field of human action recognition.However,it is still a challenge to study the continuous action recognition problem for the complexity that the continuous action recognition often compose of multiple actions and the duration of each action is indefinite.Besides,current deep learning based continuous action recognition methods,taking use of time-series annotations or clipped videos for training,are overly depending on manual annotation.For the reason above,this thesis conducts research on the task of deep learning based continuous learning recognition.The main work is listed as follows:(1)A continuous action recognition model based on Sequence to Sequence(Seq2Seq)is proposed.Seq2 seq is an end-to-end neural network model,which is developed for solving the mapping from sequence to sequence and just suitable for ours.A multi-layer Long Short-Term Memory(LSTM)network is taken to encode the input action sequence into a vector.Then the vector is decoded by another multi-layer LSTM network into the output tag sequence.The model can learn the mapping relationship between action sequences and tag sequences.Experiments on charades data set show that the Seq2 Seq model can effectively improve the performance of continuous action recognition.(2)A continuous action recognition method based on Graph Convolutional Neural Network(GCN)is proposed.Features are extracted from video clips sampled form untrimmed video and each clip is instantiated as a node in GCN.Then,the relationship between video clips is instantiated as edges for the model.The network can learn both the context information of each clip and the correlation between each video clip.A classifier is then used to identify the action of each clip,and a selector is used to detect or sort the important clips.By fusing the output of the classifier and the selector,video-level prediction result is generated.Experiments show that proposed model improves the accuracy and efficiency of continuous action recognition.(3)With the theoretical analysis and experimental verification of the algorithm,an action recognition system platform based on the Brower/Server(B/S)architecture is built.Human action videos collected by the author are taken as the system's input.By using the above algorithms,the recognition task is completed.