Research On Human Action Recognition Based On 3D Skeleton

Human action recognition has become an active research area in recent years.As it plays a significant role in fields of intelligent video surveillance,human-computer interaction and assistive technology.However,Action recognition still have many challenges in many aspects,because of the diversity of human posture,the ambiguity of action semantics and the variability of limb movement.At present,action recognition based on human joint points is considered to be a better way to express action information.It is a difficult problem that find a way to obtain the joint points of the human body from image and analyze the correlation of the joint points during the movement.In this thesis we mainly focuses on the above problems,first we propose a matching method based on non-parametric coding constraints to extract the 3D skeleton from the human body,then use the skeleton sequence to establish the space-time map of the human skeleton.The graph convolutional network is used to complete the human action classification task,finally we applied above method to the VR interaction process for realizing the application of human body action in this field.The main work contents and innovations in the thesis are as follows:(1)For the problem of extracting the 3D skeleton from the image that collected by the monocular camera,we propose a new method which is composed of two parts,first part,we extract the 2D skeleton information of the human body with using the Openpose open source,and analyze the spatial relationship between of 2D and 3D skeletons.second part,we establish a library of human posture,and the corresponding 3D skeleton information is found in the posture library by the non-parametric matching method.The whole process ensures the real-time nature of the extraction skeleton extraction process.(2)For the classification of human action task,we use the human body 3D skeleton sequence to construct the spatial-temporal map of the human skeleton,so that it fully contains the spatial-temporal information of human motion,then use the convolutional neural network to extract the feature information in the graph,so that The original problem is converted to a feature extraction problem of the graph.In this process,we construct the graph convolution kernel of the human body skeleton spatial-temporal map,and provide different partitioning strategies for the root node's receptive field sub-division.Finally,the graph convolutional neural network model was built,and it achieved good results based on the data set which made by ourselves.(3)Based on the 973 project deep space detector combined with the autonomous navigation,and the virtual reality module as the application scenario,the deep space detector VR display simulation platform is built.Innovatively we apply the human action recognition to the VR interaction process by using the human body 3D skeleton extraction,and the convolutional human action classification technology.therefore we make relatively complete closed-loop interaction mode,and discussing the human motion semantics and the general human-computer interaction process.Finally,experiments are used to confirm the feasibility and reliability of each experimental link.