The Research Of Human Action Recognition With RGBD Data

Human action recognition as a hot topic in the field of machine vision,it gets more and more attentions from the researchers and is widely applied in the many areas,such as:athletic performance analysis,intelligent security,virtual reality and so on.Most of the existing human action recognition algorithms are based on traditional RGB video data.Nevertheless,there are still many difficulties in practice,due to the influence of illumination,local occlusion,and complex backgrounds etc.In recent years,with the availability of low-cost and simple-operator Kinect depth camera,a lot of researchers propose to apply RGBD(RGB Depth)data which are collected from depth camera for human activity recognition.This paper focus on the method of human action recognition system based on RGBD video.Human action recognition system is composed of data preprocessing module,feature extraction module,feature encoding module,and classification module.The most important step of human action recognition system is to extract efficient and robust visual features to represent human action.Motivated by this,this paper proposes two-stream(Depth data and Skeleton data)based action recognition system.This paper mainly deals with the following three aspects.Firstly,for the depth data provided in RGBD video,we propose a depth feature extraction method based on gradient information and pyramid structure.Secondly,the feature extraction method based on skeleton data is proposed by collecting the relative depth information and relative position information of limb module in skeleton sequence.Thirdly,in order to extract the more robust features to represent RGBD video,depth feature and skeleton feature are added together to generate the final two-stream feature,consequently,a random decision forest(RDF)classifier is applied to mine discriminative features and recognize human actions.The RGBD data feature extraction method proposed in this paper takes into account the influence of the occlusion in the depth data and the incorrect joints positions in the skeleton information,and adopts data preprocessing and feature selecting steps.These processing steps have improved the recognition accuracy of the algorithm to a certain extent.The proposed human actions recognition system in this paper has been evaluated on three public datasets.Experimental results demonstrate that the proposed algorithm outperforms most of state-of-the-art methods,especially suitable for recognizing actions which are clearly different from each other.