Research On Long Sequence Action Recognition And Prediction

Human action recognition and prediction is key research topic in computer vision.With the gradual popularization and application of artificial intelligence technology in the training and competition of competitive sports,more new problems of human action recognition and prediction have arisen in recent years.The recognition and prediction of long sequence action is one of the most difficult issues.Most of the existing work has been focused on simple movements in daily life,and complex action with long duration have been rarely addressed.On the other hand,the training and competition of sports such as tai chi and gymnastics are in urgent need of the integration of artificial intelligence techniques to generate new dynamics.In view of this,this paper will study the problems in recognition and prediction of long sequence action and try to apply them to the training and competition of taijiquan.Long sequence action mainly refers to body movements that take longer time to complete.The technical challenges posed by long sequence actions are mainly in two aspects: 1.Difficulty in high-level feature extraction.Although deep learning-based methods can extract action features automatically.However,due to the long duration of long sequence action,the distinctive action features tend to change over time.2.High demand for predictive capabilities.The frequent limb movements of long sequence action and different human postures in different time periods require neural networks with strong data learning capabilities to predict more realistic future human actions.The main work of this paper is as follows.1.To study human action recognition by fusing causality and spatio-temporal graph convolutional networks for long sequence high-level feature extraction problem.Considering the existence of causality in human motion,this paper proposes an action recognition method that joint learn causality and spatio-temporal graph convolutional networks.The human skeletal structure is modeled as a complex system,and the coordinate sequences of joints are considered as time-series variables in the complex system to derive causality relationships.Edge weights are assigned to the skeletal graph according to the causal relationships between joints,and the weights are used as auxiliary information to enhance the graph convolutional network,weights of joints with stronger driving force in the neural network will be increased,so that the neural network can better aggregate action features and thus enhance action recognition performance of neural network.2.To study the action prediction based on GAN and adaptive graph convolution for the problem of diverse human posture changes in long sequence action.This paper proposes a human action prediction approach based on GAN and adaptive graph convolution,which takes the current human pose and action category as input and predicts continuous human pose skeleton sequences by a generator.In order to reduce the interference of widely varying actions in long sequences,a self-attention mechanism is used to calculate the correlation of frames,the most highly associated frames are selected to aggregate information and enhance the utilization of inter-frame temporal features.To make better use of the human structure information,adaptive edge weights,joint attention,and channel attention are added to the graph convolution,enabling the network to learn the human topology and feature enhancement for joints and channels of high importance to better learn the hidden distributions of skeletal data and thus make more accurate predictions of human actions.