Research On Skeleton Based Human Action Recognition

Human action recognition mainly analyzes the human motion data to determine the category of human action,which is a typical multi-classification task.It is an important foundation and prerequisite for the analysis and understanding of human movements,and has a wide range of applications in human-computer interaction,intelligent monitoring,intelligent sports,medical care and other fields,which becomes a popular research di-rection in the field of computer vision in recent years.According to different carriers of human action information,human action recognition can be divided into video based action recognition and skeleton based action recognition.The information carrier of the former is continuous RGB image sequences,while the latter is the joint 3D space coor-dinate sequences obtained by optical estimation,depth camera,motion capture device,or3 D pose estimation algorithm,which is also called skeleton data.Compared with RGB image and video data,skeleton data is more robust to noise factors such as camera shooting environment,background,lighting,and wearing of characters.The main research contents of this thesis are skeleton-based human action recogni-tion and pose estimation methods,and we also design a skeleton-based action recognition method for real-world application.First,we propose a skeleton-based action recognition network based on the non-local operation,and then propose a 3D pose estimation network based on 2D poses.Finally,an action recognition algorithm for soldier formation action in real scenes is designed.The specific works of this thesis are as follows:(1)we design a human action recognition network based on non-local operation.Firstly,non-local operation is used to model the interrelationship between joints and com-bined with temporal convolution operation to model the spatio-temporal joint relationship,and then the temporal and spatial feature fusion interaction is achieved by the spatio-temporal fusion module.This model achieve a competitive performance which is close to the state-of-the-art methods on the NTU RGB+D Skeleton dataset,and the model com-plexity is much lower than most previous methods.(2)we propose a 3D pose estimation method based on graph convolution.A multi-scale graph convolution operation is designed to aggregate joint features at different dis-tances,and a semantic classification strategy is combined to improve the fine-grained de-scription of neighbor joints.Then a body layered pooling module is further introduced to help joints get the local features of the skeleton at different levels.The proposed network has reached the best performance compared with similar works with lower computational complexity.(3)Aiming at the real-world task of soldier formation action recognition,we pro-pose an action segmentation and recognition method based on key pose frames,which can effectively perform semantic action segmentation on skeleton sequences containing a variety of different formation actions,and determine the action types of each action segment.The proposed method has good performance in practical applications.