Human Skeletal Action Recognition Based On Deep Learning

Skeleton based human action recognition has been an important subject in computer vision.And it attracts more and more attention due to its robustness to variations of illumination,scale and rotation.In the traditional skeleton action recognition methods,feature representation is hand crafted,and effective feature acquisition is usually difficult.In recent years,with the emergence and development of deep learning technology,neural networks have been widely used in skeleton action recognition.Recent studies have shown that recurrent neural network(RNN)or convolutional neural network(CNN)can effectively learn discriminative features for skeleton sequences.Although some recent deep learning based models have achieved remarkable performance,with the development of skeleton dataset:NTU RGB + D,which is more diverse and complex,there is still large space for improving the deep learning based model.Therefore,we will present three improved deep learning based models for skeletal action recognition in this paper.We first propose a time selection long short-term memory(TS-LSTM).Through the combination of LSTM and max pooling,the variable length context dependence can be learned.Then,a spatio-temporal convolution neural network(ST-CNN)is proposed,it's a body parts based model.Through the special parameters setting of two temporal convolutional layer,the discriminative spatio-temporal features can be learned.Finally,we propose an attention network,joints selection convolutional neural network(JS-CNN).Through its joints selection module and frames selection module,the discriminative joints and frames can be automatically selected,then more discriminative features can be obtained.We evaluate our three methods on four datasets,and our methods all achieve state-of-the-art performance compared to recent models.Besides,through the extensive experimental analysis,the effectiveness of the proposed models has been demonstrated.