Human action recognition is a very important research direction in the fields of computer vision and pattern recognition.The emergence and popularity of consumer-grade depth sensors(such as Microsoft Kinect)have made extensive attention to the use of deep data for human motion recognition.The depth data is robust to the change of illumination,the clutter of the background and the change of the viewpoint,which make up for the deficiency of the traditional human action recognition method.These advantages provide a lot of new ideas for the study of human motion recognition.Although many descriptors of action based on depth data have achieved considerable results,how to extract the characteristics of human action based on depth data is still an open problem to be solved.The action characteristics have relatively complex data and the high dimensions,designing an efficient classification algorithm is also an urgent problem to be studied.For the currently widely used scheme of using deep learning technology to automatically learn features,due to the limited number of samples,it has not been able to achieve good results in the field of human action recognition.Facing these problems,this paper proposes two different human action recognition schemes based on different dimensions of deep data feature fusion based on previous researches.Considering the complementarity between different features from the perspectives of 3D depth data and 4D depth data,the features are combined effectively and the suitable classification method of human action recognition is explored,so as to achieve better recognition results.The main work and results of this paper are as follows:Firstly,it sorts and analyzes the related researches of human action recognition based on depth data,and summarizes the research status of human action recognition based on depth data from the feature extraction,the classification method and the human motion recognition methods based on feature fusion.The types of deep data,common feature representations and classification methods are discussed comprehensively to form an overall research idea.The characteristics of 3D contour features,spatio-temporal interest points,skeleton joint points and point cloud information are explored,and the principles of five algorithms including DTW,KNN,SVM,ELM and Softmax are analyzed for exploring the deep feature representation and the action classification,which suitable for the topic of this paper.Secondly,a method for human action recognition based on 3D depth data feature fusion is proposed.In this method,a human contour feature method based on wavelet transform is proposed.Then K-means clustering is used to find the most representative feature values.Finally,the DTW algorithm is used to match the actions.The classification results verify that our proposed contour descriptor has a certain descriptive power.Then,the spatio-temporal interest point feature extracted from the depth images and the joint data from the human skeletons are merged,and the three features are directly connected in series to perform PCA for achieving feature fusion,and finally SVM is used for classification.The effectiveness of the proposed method is verified by comparative experiments.Thirdly,a human action recognition method based on 4D depth feature data fusion is proposed.In this method,an improved energy-based spatio-temporal pyramid is proposed to obtain spatio-temporal information of deep action sequences.The extracted 4D normal features based on low and high levels are merged.The 4D features are mainly 4D surface normal direction features and super normal vector features.Super normal vector features,capture local motion and geometric information at the same time,can be considered as an advanced feature evolved by low-level feature.According to the proposed energy-based spatio-temporal pyramid method,the final feature obtained by fusion has stronger description ability.Finally,BLS is used to classify actions.The method has achieved good recognition results by conducting different contrast experiments on three open classical depth datasets and small samples in various cases.Fourthly,aiming at the slow training speed of the classification algorithm in the human action recognition method based on 4D depth data feature fusion,a fast broad learning algorithm based on matrix decomposition is proposed.In the original BLS,as the mapping nodes and the enhancement nodes increase,the training time will increase.But the recognition rate of the actions may decrease.Aiming at this problem,the improved broad learning algorithm proposed in this paper utilizes the knowledge of matrix decomposition to effectively decompose the output layer weight of the network and obtain a new calculation method,thus reduces the training time.It is verified on the three datasets that the proposed algorithm still has advantages in training time under the premise of ensuring that the recognition rate does not decrease. |