Research On Human Action Recognition Method Based On Skeleton

With the rapid development of the field of computer vision,human action recognition has attracted the attention of many researchers.This research direction mostly uses video analysis methods to recognize human actions.However,the flexibility of human actions is high and the differences are small.There are still many challenges in using video analysis to accurately identify the types of human actions.With the increasing maturity of video acquisition equipment,human skeleton information in sports is becoming more and more accessible,and human skeleton information is robust to problems such as lighting,venues,and occlusion.Therefore,video-based human action recognition is gradually changing It is a skeleton-based human action recognition.The three-dimensional coordinate data of the human skeleton records the activities of people in various situations such as life,work,and entertainment.The research on action recognition will greatly promote the development of human science and technology civilization,facilitate people's social life,and be applied to intelligence Nursing,security monitoring,sports fitness and social platform content review and other aspects.Skeleton-based human action recognition mainly studies how to model human skeleton information,extract effective temporal and spatial information from it,and then achieve the purpose of recognizing human actions.In this paper,human action recognition based on human skeleton information is the research content,and the following research work has been completed:(1)Human body action recognition algorithm based on Skele Motion-ResNeXt(SM-RNXt)network is proposed.First,the human skeleton of each action is modeled on the action structure,the action size and direction characteristics are calculated according to the action structure,and finally the two are merged to obtain the Skele Motion skeleton image with spatio-temporal information.Then input it into the ResNeXt-50 network to extract spatio-temporal features to achieve the purpose of identifying and classifying actions based on human skeleton.(2)On the basis of the SM-RNXt network,the graph convolution is combined to propose a human action recognition algorithm based on the two-stream network ResNeXt-GCN(RNXt-GCN).The skeleton of the human body is composed of 25 main joint points and the edges between them,which is especially suitable for classification and recognition by spatio-temporal graph convolution network(ST-GCN).However,the spatio-temporal graph constructed by the ST-GCN network can only learn the single temporal information between different frames of the same joint,and cannot effectively learn the temporal information between similar actions.The Skele Motion skeleton image can effectively learn the temporal information between similar actions through multi-scale modeling of human skeleton information,thereby making up for this shortcoming of ST-GCN.The network combines the temporal and spatial features extracted by the two to obtain the final human action recognition and classification results.And the network was tested on UTD-MHAD dataset,Northwestern-UCLA dataset and NTU RGB+D dataset.On the UTD-MHAD dataset and Northwestern-UCLA dataset,the RNXt-GCN network has obtained better results than the baseline network;on the NTU RGB+D dataset,the RNXt-GCN network experimental results have been competitive result.(3)Propose a human body action recognition algorithm based on Skele Motion Reference Joints Image-ResNeXt(SMRJI-RNXt)network.Tree Structure Reference Joints Image(TSRJI)skeleton image models the human skeleton information by using two technologies: reference joints and depth-first tree traversal,effectively extracting the spatial information in the human skeleton,and inputting it to the ResNeXt-50 network for feature extraction,and extracting the resulting spatio-temporal features The spatio-temporal features extracted from Skele Motion bone images were fused,and further experiments were carried out on the NTU RGB+D 60 data set and NTU RGB+D 120 data set.The action recognition accuracy is higher than that of the SM-RNXt network,which proves that the TSRJI skeleton image The extracted action features have a certain supplement to the skeleton features extracted from Skele Motion skeleton images.In order to further prove the effectiveness of the network,this paper integrates it with the ST-GCN network,and the accuracy of action recognition on the NTU RGB+D 60 dataset and NTU RGB+D120 dataset has been further improved,which are both higher than The RNXt-GCN network proves that the SMRJI-RNXt network can provide more effective spatio-temporal features for action recognition.