Research On Human Action Recognition Based On Deep Learning

As a hot topic in the field of computer vision,artificial intelligence and pattern recognition,human action recognition has attracted more and more attention.It is widely used in video surveillance,intelligent robotics,human-computer interaction,game control and multimedia video retrieval and so on.However,owing to its vulnerable to self-occlusion,perspective,light,dynamic background and other factors,human action recognition is still a challenging task.To overcome the interference of external factors,this paper extracts the depth information and skeleton information of human actions for human action recognition.Moreover,to avoid the problem of poor data processing ability of traditional machine learning method,this paper introduces deep learning for human action recognition.Deep learning has strong nonlinear fitting ability,feature representation ability,and high-dimensional data processing ability.Based on depth information and skeleton information,we proposed the methods of using deep learning for human action recognition.Finally,we built a human action library for virtual learning environment(CQ-VLAction)to test relevant learning algorithms.The main research contents of this article are as follows:Firstly,using skeleton information,a human action recognition method based on deep belief network(DBN)was represented.The proposed method extracts the static features,motion features and the offset as the effective skeleton features.And then,an optimized DBN was used to recognize the skeleton features.At the same time,the number of joints was changed to verify its effect on human action recognition.Secondly,the method of human action recognition based on the improved convolutional neural network(ICNN)was studied.We use depth motion maps(DMM)to extract the depth features of depth information,and then apply the proposed ICNN to recognize the features.The proposed ICNN model uses three-dimensional input and two-dimensional process identification to speed up the computation and reduce the complexity of recognition process.We evaluate our approach on two public 3D action datasets: MSR Action3 D Dataset and UTKinect-Action Dataset.Experimental results show that the proposed methods of human action recognition achieve higher average recognition rate of 91.3% on MSR Action3 D dataset,and 97.98% on UTKinect-Action Dataset,which are more effective than most existing methods.Furthermore,based on the depth feature and skeleton feature,we used the proposed ICNN to recognize the human actions and obtain the best recognition performance and robustness.Finally,we built human action database for virtual learning environment(CQ-VLAction)using Kinect and Matlab tool.We further test the proposed ICNN on our CQ-VLAction.Experimental results show that ICNN can be easily generalized to different datasets and has better recognition performance and robustness.In addition,our CQ-VLAction database has also laid a good foundation for our further research on the natural interaction application of virtual learning environment based on human action recognition in next years.