Design And Implementation Of Human Action Recognition System Based On Skeletal Data

With the continuous development of computer vision and image processing,human action recognition technology has been widely used in the fields of human-computer interaction and virtual reality.However,due to the complexity of human movement and to the variety among the same actions performed by distinct subjects,human action recognition can still be a challenging task.As a supplement of color camera,depth camera has made a great breakthrough in research and development,and some depth sensor devices can provide real-time human skeleton data.Therefore,human motion recognition based on skeletal data has become a research hotspot.In this paper,based on the skeleton data provided by Kinect,the characterization of the skeleton data and the classification and recognition algorithm are deeply studied.A new framework of human action recognition is proposed and validated by experiments.Then the framework is used to design and implement a human action recognition system.The main work and innovation of this paper are as follows:1.This paper proposes a joint feature extraction method based on joint segmentation.The method fully studies the characteristics of skeletal data and human movement.Firstly,the human skeleton is divided into three parts:the trunk,the first level joint and the second level joint.Based on the three methods of joint angle,joint displacement vector and joint relative position.The feature extraction of skeletal data is carried out separately,and a feature set composed of a plurality of feature subsets is constructed.This method not only eliminates a lot of redundancy in skeletal data,but also retains the features of strong discriminant ability.In the experiment,it verifies that each feature subset can effectively improve the recognition accuracy.2.The Vector of Locally Aggregated Descriptors(VLAD)is used to characterize the motion and the VLAD algorithm based on PCA whitening is proposed.Different from the commonly used HMM model and DTW algorithm in the field,the VLAD model which is used in the field of image retrieval is applied to the motion recognition.Meanwhile,VLAD can also combine spatial and temporal feature of skeletal data.Afterwards,the PCA whitening method was used to improve the VLAD,which reduced the data dimension and removed the noise and redundancy at the same time,thus improving the frame recognition accuracy.3.The Large Margin Nearest Neighbor algorithm(LMNN)is used to construct classifier,and the problem of over-fitting and the computing speed have been improved.Based on the above feature description method,this paper uses the LMNN algorithm to improve the recognition accuracy of the nearest neighbor algorithm(KNN)through the metric learning method.Then,in this paper,aiming at the over fitting problem which is easy to occur in action recognition,a regular item is introduced in LMNN,and its operation rate is improved by using mini-batch gradient descent.Finally,the performance of the improved LMNN algorithm is verified experimentally,and compared with the SVM algorithm.4.Design and implement a motion recognition system.Motion recognition system is mainly composed of motion capture module,motion segmentation module,motion training and recognition module.Among them,in the motion segmentation module,in order to segment the motion sequence captured by the camera better,a method based on motion state and time threshold is proposed.