Human Action Recognition Based On Depth Maps And Skeleton Data

In computer vision and pattern recognition, human action recognition is an active branch. Earlier attempts at action recognition have involved using video sequences captured by video cameras. With imaging techniques advance, the introduction of cost-effective depth cameras, e.g. the Microsoft Kinect, facilitates a variety of visual recognition tasks including action recognition. Compared with RGB images, depth date provide 3D depth data, which is insensitive to changes in lighting conditions and ignores the color variability induced by clothing, skin, hair and background.This paper has been obtained many quick and effective methods based on depth maps acquired by Kinect. The main contents of this paper are as follows:First, in order to solve the problems that the high complexity of traditional color video-based action recognition algorithms and cannot recognize the action in real time. This paper proposes a real-time and effective method of human action recognition. The algorithm firstly obtain 15 skeleton joints from the 20 skeleton joints acquired by Kinect. Secondly we structured location model, motion model and angel model. Then we added these models together and evaluated these models by HMM classifier. The proposed method was tested on MSR Action3 D dataset and the experimental results demonstrate that the proposed method has a fast speed and better recognition, also it meet the real-time requirement in the depth video sequence system basically.Second, the traditional color video-based action recognition algorithms still cannot deal with perfect recognition for the insufficient of two-dimensional information and cost high. In order to solve the problems, a novel human action recognition method based on three-dimensional image(depth image)sequence was put forward. On the temporal dimension, Temporal Depth Motion Maps(TDMM) was proposed to describe the action. Specially, the entire depth maps were divided into several sub-actions under three orthogonal Cartesian planes. The absolute difference between two consecutive projected maps was accumulated to form a depth motion map to describe the dynamic feature of an action. Support Vector Machine and Regularized Collaborative Representation were used to classify the proposed descriptors at last. The proposed method was tested on two authoritative datasets: MSR Action3 D dataset and MSRGesture3 D dataset. The experimental results demonstrate that the proposed method has a fast speed and better recognition, also it meet the real-time requirement.We proposed Temporal Depth Model(TDM) based on Temporal Depth Motion Maps(TDMM). On the spatial-dimension, Spatial Pyramid Histograms of Oriented Gradient(SPHOG) was computed from the TDM for the representation of an action to obtained a novel descriptor, just as TDM-SPHOG descriptor. Also we applied Support Vector Machine and Regularized Collaborative Representation to classify the proposed descriptor. We tested our descriptor on MSR Action3 D dataset and MSRGesture3 D dataset. The experiments on this descriptor were outperforming the existing methods at recognition accuracy.The methods proposed in this paper based on depth data were outperforming the major existing methods at recognition accuracy. Also it can meet the real-time requirement. All of these can demonstrates the effectiveness of our proposed methods.