Research And Implementation Of Action Recognition Based On Pose And Skeleton

In recent years,with the upgrade and application of high-definition video equipment technology,the action recognition technology has been implemented and applied in many areas such as smart cities,public security,and smart home,and it also allows the research of action recognition technology to be further improved.Traditional action recognition algorithms usually adopt manual design feature extraction methods,such as three-dimensional modeling,template sequence matching and other methods.Although the recognition rate of the traditional method is still acceptable,its workload is large,and the manual feature extraction method is complex in design and robust.In addition,there are many factors such as occlusion,illumination,camera angle change,and movement interference in the real scene.Although the traditional method is suitable for the action recognition in the simple scene,it is difficult to recognize in the real scene and the accuracy is limited.The purpose of this paper is to study the action recognition method that combines video sequences and skeleton information.According to the different characteristics of image sequence features and skeleton sequence features,different deep neural networks are used respectively,and two network models are proposed for different fusion methods.(1)The pose-based action recognition method is studied.Using the pose estimation method based on the part affinity fields algorithm to segment the human model and extract the joint information,the branch structure convolutional neural network is used to learn and train the optical flow information and appearance characteristics of different parts,and then the network extraction is performed.The features of the network use dynamic and static feature aggregation method to obtain the spatial and temporal features of the video sequence,then predict the feature classification.(2)The method of action recognition based on skeleton information and pose information is studied.For skeleton sequence feature extraction,two feature extraction methods based on recurrent neural network and graph convolutional network are implemented.Two models are proposed for the action recognition method based on the fusion of skeleton information and pose information:the first model is a combination of a pose-based convolutional neural network and a skeleton-based recurrent neural network;the second model is a combination of a pose-based convolutional neural network and a skeleton-based graph convolutional network.Both models use the skeleton sequence and video sequence as input,acquire the features of spatial-temporal information through the network model,classify and predict the features,and fuse the class scores of different networks to obtain the final classification prediction results.Through experimental verification,the method based on the second model has higher accuracy and better performance.