Research On 3D Skeleton Action Recognition Based On Deep Learning

As an important part of the field of computer vision,human action recognition has a wide range of application prospects in various fields,such as intelligent monitoring,human-computer interaction and so on,which makes it a hot research topic in today's society.Especially with the appearance of Kinect,more and more high-dimensional information can be expressed.Action recognition based on 3D skeleton attracts many researchers to study.The 3D skeleton extracted from the depth image by the depth camera is robust to the changes of light,appearance and perspective.At the same time,the maturity of skeleton real-time estimation algorithm further promotes the development of this research.Early 3D skeleton based action recognition mainly used hand-made features for action recognition.With the development of deep learning technology,the action recognition method based on deep learning has been successfully applied in the research of3 D skeleton action recognition.Although some deep learning network models proposed for 3D skeleton action recognition have achieved good performance,there is still a large improvement space for different scenarios.In this paper,two 3D skeleton action recognition methods based on deep learning are proposed,which aim at the action recognition under two person interactive scenarios and different imaging conditions such as camera angle change scenarios,in order to achieve a higher recognition rate.(1)In this paper,a model based on geometric features combined with LSTM network is proposed for 3D skeleton action recognition.In view of the 3D skeleton action recognition in the context of two person interaction,it is easier to find the richer information contained in two person interaction action than that in single person action by taking into account the geometric characteristics of the skeleton sequence.The feature vector of skeleton geometric features is selected to replace the 3D coordinates of skeleton joints as the network input,and the geometric features and deep learning method are combined.In addition,a time sequence selection LSTM network is introduced,which has the ability to select the most recognizable time segment features.It can remove redundantsequence frames and reduce the amount of computation.Finally,the method is tested on SBU interaction dataset and UT Kinect dataset.Experiments show that the recognition rate of the model in SBU interaction dataset is 99.36%,which is significantly higher than most of the previous methods using joint coordinates as network input.The experimental results show that the method is effective.(2)In this paper,a convolution neural network based on the time sequence diagram of spatial features is proposed for 3D skeleton action recognition.In view of the different imaging conditions such as the camera angle,we propose the time sequence diagram of spatial features to describe the skeleton considering the view invariant feature of the skeleton.The skeleton distance and the skeleton angle are calculated by the skeleton position information,and then the skeleton spatial features are described by combining the two information,and arranged by time dimension to form the time sequence diagram of spatial features.Then it is input to convolutional neural network model for action recognition and classification.Finally,the method is tested on the standard dataset Berkeley MHAD and NTU RGB + D.Experiments show that the recognition rate is84.12% in cross-view test on NTU RGB + D dataset,which is higher than that of most previous methods.The experimental results show that the method is effective.