Human Action Recognition Based On Depth Data Feature Fusion

As a popular research field,Human action recognition has provided human with great convenience for the wide application in our daily life.A large number of researchers are dedicated to the research of this topic.More and more methods have been proposed and achieved good results.In recent years,with the release of Microsoft Kinect,it provides an effective and inexpensive way to obtain deep images.Compared to traditional RGB images,depth maps are insensitive to the external environment and can provide 3D structure information,which provides a new way for the study of action recognition.Although many feature descriptors based on depth maps have achieved rather good results,the single type of feature is often not comprehensive enough to describe an action.To solve this problem,we proposes different feature fusion strategies and uses features complementarity to achieve better recognition accuracy.The main work of this paper are as follows:Firstly,the global feature and the local feature are fused.The 3D motion trail model(3DMTM)is extracted from the depth image sequence,then the multi-layer gradient direction histogram(MHOG)is extracted to obtain the local structural features of the motion,but it lacks the grasp of the global characteristics of the motion.To solve this problem,we propose to add Gist to get the global information of actions,and combine global and local features for action recognition.The simulation results on public databases verify the effectiveness of the integration of global and local features.Secondly,the temporal features and spatial features are fused.First of all,a feature descriptor which can better describe the temporal characteristics of actions is presented in this paper.The descriptor is obtained by applying twice gradient direction histogram(Histogram of Oriented Gradients,HOG)to the sequence of actions at different Pyramid levels,which is called PHOG~2.Due to the human action is a continuous three-dimensional space structure in time domain,PHOG~2 is short of the characterization for space information of actions.So,the Local Binary Pattern(LBP)is extracted from the depth motion maps(DMM)to get DMM-LBP descriptor,which is used to describe the spatial features of actions.Then it is fused with PHOG~2using canonical correlation analysis(CCA).The result of fusion is classified utilizing linear SVM.The high recognition rate has been achieved on two public databases.