Human Action Recognition Based On Depth Images

Human action recognition is a widely studied field in computer vision, machine learning and image process. It is well known that Human movement is an articulated system, while it is hard to capture human joint location for traditional action recognition methods based on RGB channels. Meanwhile, cluttered background, occlusions, viewpoint changes and varying illumination conditions will result in recognition performance decrease significantly. With the recent release of Kinect sensor and technology assessing skeleton joint position from depth image matured,3D skeleton joint position information from depth images is widely used as human body representation and has achieved good recognition performance.A suitable representation of human posture extracted from image sequences plays an important role in human action recognition. Previous action information descriptors usually use3D joint position evaluated from depth image sequences directly, but large amounts of information redundancy are also brought out. Refer to background subtraction technique, a novel representation of human posture is proposed in this paper. It is well known that human action is composed of ordered posture set, and the difference among postures is only few3D joint positions, most of3D joints position usually changes a little. In this paper, the3D joint position of first frame is regarded as the reference frame, the difference of3D joint positions between the rest frame of the action sequence and the reference frame is defined as a complete action feature descriptor. The proposed feature representation reduces the data redundancy and impact of human motion style while action recognizing, which also has the advantages of shift-invariant and view-invariant.In generally, high computational complexity is generated when matching among actions in high-dimension space, while manifold learning method can be used to embed motion data effectively into low-dimensional hidden space smoothly and to obtain low-dimension motion model. Therefore, a manifold-based framework is presented for human action recognition using depth image data captured via depth camera. In training phase, Lapacian Eigenmaps is introduced to build action model in low dimension space. In test phase, nearest-neighbor interpolation technique is applied to map test sequence to manifold space, and then a novel modified Hausdorff distance is employed to measure similarity and fitness between test sequence and train data in matching process.Action sequence is an ordered set of postures. Therefore, considering timing constraints of the relationship among postures, a new type of action recognition method based on local window matching is proposed in this paper. Compared with traditional Bag-of-Words, the proposed method has many improvements including model feature learning, feature quantification, object describe. In training phase, all feature words are trained individually from each action, which is different from traditional global training. In descriptor quantization processing, a novel linear mapping means with local self-adaptive is also proposed in this paper, which is used to replace for the conventional quantization way. In the stage of action sequence character representation, a whole action sequence is divided into several segments, local window matching strategy has been adopted to describe action segments. Two different kinds of local window matching strategy are proposed in this paper, which is temporal pyramid and sliding window. Finally, histogram intersection is used to figure out the similarity of two action sequences in testing process.Based on two types of proposed methods, an action recognition system using Kinect sensor is designed and has achieved good performance. This paper has also tested proposed methods on the MSR Action3D dataset and achieved state of the art accuracy compared to the related work. The experiment results show that the proposed method is effective for human action recognition based on depth image sequence.