Research On Skeleton Based Action Recognition

Action recognition is one of the most challenging research directions in the field of artificial intelligence and computer vision.And it is widely used in human-computer interaction,healthcare,intelligent security and intelligent robotics.This approach is mainly implemented by aggregating predictive information from each frame of a video clip.Although frame-based classification has been proven to be effective,it is still difficult to extend from the twodimensional image domain to the three-dimensional video domain.The technique requires not only a huge increase in computational cost,but also the need to capture the spatio-temporal context relationship between different frames.Therefore,how to effectively extract the information from video clips is a key point in action recognition domain.Compared with the traditional RGB information,the skeleton sequence does not contain RGB information and has the advantages of simplicity,easy calibration,and easy calculation,which makes it be popular in the field of computer vision.How to extract the features in the skeleton sequence and use them have been the most critical part in skeleton based action recognition.This technology uses graph convolutional neural networks to extract features from non-European structure data,learn potential information about related behaviors,and obtain the overall distribution of samples by learning prior knowledge.On this basis,this dissertation proposes three action recognition methods based on human skeleton.The specific research results and contents are as follows:(1)From the perspective of improving recognition accuracy,we propose multi-directional image convolution action recognition method based on human skeleton.First,we preprocess Multi-Directional-Motion Based Graph Convolutional Network.Second,we use the weighted average method of symmetric normalization to construct a spatio-temporal map,so as to capture the spatiotemporal characteristics of the bone data more effectively.Finally,we fuse the joint point information flow and the bone length information flow to obtain the final classification result.(2)From the viewpoint of reducing the number of parameters in the network,a multimodal lightweight graph convolution human skeleton action recognition method is proposed.Firstly,the multi-modal data fusion method is used to fuse multiple information flow data.Secondly,the corresponding spatial information and time information are obtained through the spatial flow module and the time flow module.Finally,the fully connected layer is used to obtain the final classification result.(3)From the aspect of improving the recognition accuracy as much as possible and reducing the amount of calculation,a lightweight shift map convolution action recognition method based on attention mechanism is proposed.First,we preprocess the skeleton data to generate information flow data of human skeleton joints that can be directly used for training,bone length information flow data,joint point information flow data based on motion information,and bone length point information based on motion information Streaming data sets.Secondly,the spatial and temporal information of the fused data are obtained through the spatial shift module and time shift module based on the attention mechanism.Finally,the joint information and bone information as well as their corresponding motion information are integrated in a multi-stream framework to obtain the final classification result.