Human Skeleton Action Recognition Based On Deep Learning

With the development of modern information technology,video information acquisition equipment has been widely used.Computer vision research based on image,video,and skeleton datasets has developed rapidly.In particular,human action recognition based on deep learning methods has gradually become a research hotspot in the field of computer vision,and has wide application value in fields such as automatic driving,gesture control,and human-computer interaction.Human action recognition based on skeleton dataset is more robust than video image human behavior recognition method,and is not affected by factors such as lighting,occlusion and background color mixing.More and more researchers have invested in action recognition based on human skeleton data and achieved remarkable results.This paper proposes three different models based on deep learning human skeleton dataset: a network model based on two-stream convolutional neural network,a model based on the combination of convolution neural network and recurrent neural network,and a model based on multi-stream convolutional neural network.(1)Aiming at the problem that the traditional convolutional neural network model has insufficient extraction of action information,this paper proposes a human skeleton action recognition network model based on the dual-stream convolutional neural network based on the dual-stream structure model.The original skeleton sequence is used as input data of a branch,and the attention mechanism enhances the skeleton sequence or the skeleton time-domain differential sequence as input data of another branch.The dual-stream convolutional neural network fully extracts the action feature information from the input data of the two tributaries,and selects the appropriate fusion method for feature fusion.It proves that the dual-stream feature fusion convolutional network model is of great help to the improvement of the detection results.(2)Aiming at the problem that the recurrent neural network pays too much attention to the time dependence of the skeleton,but for the insufficient extraction of the skeleton information of the spatial structure,the method of combining the recurrent neural network and the convolutional neural network is used to propose a combination of convolution and recurrent neural network.Human skeleton action recognition network model.The time module uses Long and short term memory neural network(LSTM)nerves to extract time-series features from the three branches of the whole,part,and details.The space module consists of convolutional neural networks to extract spatial features.Successfully solved the problem of insufficient spatial information feature extraction.(3)Aiming at the problem of insufficient feature extraction of spatiotemporal cooccurrence information of human skeleton sequences,this paper uses the methods of time-domain difference and space-domain difference to describe the skeleton spacetime features.Combined with the setting method of graph convolution network for convolution kernel,a human skeleton action recognition model based on multi-stream convolution neural network is proposed.This model is based on the classic hierarchical co-occurrence network model.Firstly,the skeleton movement is newly modeled,and then the network model is designed for multi-tributary design and feature fusion.The convolutional neural network of each stream extracts the time feature information,space feature information and spatiotemporal co-occurrence feature information of the skeleton action from the original skeleton data,time-domain difference and spacedomain difference data,and selects the appropriate fusion method for feature fusion.For the above three models were verified on the skeleton dataset,through a large number of experimental analysis and comparison with other advanced network models to prove the effectiveness and advanced nature of the model.