Research On Human Action Recognition Based On Spatio-temporal Graph Convolutional Neural Network

The human action recognition is essential for understanding the semantic representation of the actor and inferring the behavior intention of others.With the development of visual sensors such as Microsoft Kinect and Intel RealSense,the acquisition of 3D structural action information of the human body,how to effectively model human skeletons,extract discriminative features of human movements,and improve the accuracy of motion recognition are urgently needed research questions.In our work,the method of human action recognition based on spatio-temporal graph convolution neural network is studied.By constructing a spatio-temporal of human skeleton structure and extracting spatio-temporal features of human actions,the model based on the spatio-temporal graph convolutional neural network method in different scenarios is improved,and the human action recognition system based on spatio-temporal graph convolutional neural network is constructed.The specific research work is as follows:1.Our work focuses on the research of co-occurrence of human skeleton actions and skeletal joints in human action recognition,and constructs a spatio-temporal cooccurrence graph convolutional neural network.This network combines manual distribution of human body parts and modeling based on the natural articulated skeleton topology graph between bones and joint points.Aggregate node-level and graph-level cooccurrence features,and further use the spatio-temporal graph convolutional neural network and joint co-occurrence features to model the spatial configuration and temporal dynamics of the skeletal sequence,effectively learn the discriminative features of human action recognition.2.A novel directed graph neural network is designed to extract the action information propagated in adjacent joints and bones according to the kinematic correlation between human joints and bones and the defects of human body representation in the original skeleton topology graph.At the same time,this work proposes a spatio-temporal adaptive directed graph to adapt to the topological structure of some specific actions.In addition,the time difference of bone information is also important for skeleton-based action recognition.3.The human skeleton data of the same action changes greatly due to different viewpoints.We construct a directed spatio-temporal graph convolution based on adaptive viewpoint and LSTM network combined model(AV-DGCN-LSTM),and capture features such as the spatial position representation of the skeleton joints and dynamic changes in time based on the feature information of the bone edges at the best viewpoint.The recognition accuracy on the NTU RGB + D dataset is 89.33%(Cross Subject,CS)and 94.27%(Cross View,CV),and the recognition accuracy on the SYSU-3D Human-Object Interaction dataset is 86.18%(setting-1)and 87.61%(setting-2).The recognition accuracy on the SBU Kinect Interaction dataset is 97.64%,and the recognition accuracy on the MSR Daily Activity 3D dataset is 97.23%.4.We design and implement a human action recognition system based on spatiotemporal graph convolutional neural network.This system is implemented through data acquisition platforms such as OpenPose and OpenCV,based on development tools and scientific computing architecture such as python,Tensorflow.Functional modules include data acquisition,pose estimation,pedestrian tracking,action recognition,and more.These modules respectively implement the collection and processing of visual data,the extraction of human skeleton nodes,body posture tracking and human skeleton action recognition for single and multiple people.In real scenarios,the system can realize realtime human pose estimation,tracking,and motion recognition,which validates our proposed human motion recognition methods.