Action Recognition Based On Convolutional Neural Network

Human action recognition has been a very important topic in computer vision.Its purpose is to detect and clarify the action of the subject from the videos,so that the computer can understand the human actions as well as the deep meaning of the scene.Human action recognition is widely used in many fields and of great research significance,as intelligent control and human-computer interaction.There are many existing methods to improve the accuracy of Human actions,but the differences between action classes,the difference within classes,and the background interference which all lead to the difficulty of extraction and recognition of motion features.In this paper,an effective yet simple video representation is proposed for RGB-D based action recognition.The depth map sequence is represented into three structured dynamic images at body,part and joint level of granularities through hierarchical bidirectional rank pooling.Different from previous works that applied one Convolutional Neural Network(ConvNet)for each part/joint separately,in this paper the depth sequence at each granularity level is organized into one structured motion image as the input of a ConvNet,based on the proposed effective Mask-based resizing method.The structured motion images not only preserve the spatial-temporal information but also enhance the structural information of the depth sequence,and at the same time minimize the computing and storage requirements.Next the three kinds of structured motion images are fed into ConvNets respectively,and the final result is built on multiplication fusion.This method aggregates spatial –temporal structural information in a global to local manner both in spatial and temporal domains for action recognition,as well as take advantage of ConvNets.The proposed method was evaluated on five benchmark datasets,namely,MSRAction3 D,G3D,MSRDailyActivity3 D,SYSU 3D HOI and UTD-MHAD datasets and achieved the state-of-the-art results on all five datasets.