Research On 3D Skeleton-Based Human Action Recognition With Deep Learning

Human action recognition has a wide range of application scenarios and is a hot research direction in many fields.With the advent of RGB-D sensors,action recognition has been extended from two-dimensional space to three-dimensional space.3D skeleton has been studied with its highly abstract expression of action.This paper will take the deep learning method as the carrier to carry out in-depth research on the spatial feature expression,time motion association,spatiotemporal dependence and discriminant feature extraction of 3D skeleton sequences.Experiments are carried out on NTU-RGB+D data sets and MSR Action 3D data sets.The main research work is as follows:To solve the problem of RNN spatial feature expression and gradient in training,a spatial enhancement Res-Ind RNN model is proposed.Firstly,the spatial feature representation of 3D skeleton sequences is enhanced by angle extension and geometric transformation.And then input it into the Res-Ind RNN to learn the relationship between temporal dynamic association and spatial structure.The experimental results show that the method has better performance than RNN and LSTM.This paper presents a DA-AGCN model to solve the problem that the skeleton graph fixation only represents the physical structure of the human body in the GCN.First,the global graph structure and association graph structure are proposed to learn the connection relationship between joint points with AGCN.Then the DAT module composed of JAM and GATP is proposed to distinguish the classification contribution of each joint.The experimental results show that the method can effectively use the topology of 3D skeleton and learn the characteristics of important joints,which significantly improves the recognition rate of 3D skeleton action recognition in human body.Aiming at the problem that most methods only use skeleton first-order information and lack high-level information utilization,a M-IAGCN-MAM model is proposed.First,three kinds of high-level semantic features of joint,skeleton and motion are inputted into the network according to three streams and fused into first-class in the early stage.Then the JAM is extended to capture the MAM,of spatiotemporal dependencies and the time convolution layer to a multi-kernel parallel structure.Finally,the residual connection between modules is modified to block residual connection.Experimental results show that the model can effectively utilize the spatiotemporal dependence of multi-scale motion information learning action and further improve the recognition rate of human 3D skeleton action.