Research On Human Action Recognition Based On 3D Skeleton Sequence

As a new research direction in computer vision and augmented reality, human action recognition is of great theoretical value. In the application field, such as smart home, behavior analysis, game entertainment and medical rehabilitation, human action recognition has also played a central role. Before most of the human action recognition researches have been carried out in the video sequences and all kinds of video analysis algorithms have been designed, because of the effect of the complex background, Illumination change and occlusion, the human action recognition has many difficulties, the related applications have been greatly limited. However, with the emergence of a low-cost depth visual acquisition device which can easily extract more discriminative and compressed 3D human skeleton data in real-time, the problems existing in the recognition based on video can be solved, human action recognition ushers in a new light.Our research on human action recognition bases on human 3D skeleton sequence extracted from human body using Kinect. Some essential technologies, such as feature representation, the establishment of high-level model and the similarity measure between the model and the feature sequence, are explored and achieve good results in the human action recognition. A high-level model is established to describe the characteristics of the sequences, which can solve the temporal dynamic problem of the alignment between feature sequences. The main works and contribution of this thesis are carried out around this point and can be concluded as follows:1. A human action recognition method based on a hierarchical modelThe action class is divided into several groups by hierarchical structure which can turn a complex task into several simple subtask. In the hierarchical model, all the actions can be divided into several groups according to the movement of the human body. In each group, the pre-trained SVM classifier can perform labeling on the features extracted from skeleton sequence. We use feature trajectories linked with posture features to represent actions. To measure the similarity between two feature trajectories with different lengths, anisotropic diffusion filter is introduced to smooth the features and Fourier Temporal Pyramid is used to extract the frequencies as the final features. Because of the use of frequency information as features, the algorithm can recognize the repeated and incomplete actions. Experimental results show that this approach can achieve better accuracy than previous methods.2. A real-time human action recognition method based on the vector spaceRepresenting actions as vector space can solve the temporal dynamic problem of trajectory alignment. Novel spatio-temporal features based on human skeleton sequence is proposed to describe motion information and relative position information. Each frame feature extracted from each skeleton is used as one point in vector space, then one action sequence is represented by a point set. To recognize actions in real-time, the Kmeans approach is used to cluster features and the clustering centers are applied for representing actions. The way of t using vector space to describe actions can solve the problem existing in the actions with periodic, incomplete and temporal dynamic characteristics. Our algorithm can recognize actions in real-time and increase the recognition accuracy effectively in two newest action datasets.3. A human action recognition approach based on weighted graphs and global optimal similarity measuringWeighted graph is proposed to describe actions for the first time. The vertices of weighted graph is the clustering centers which is extracted from the frame feature set of each class. The temporal correlations between two arbitrary vertices are used as the edges of weighted graph. A time clustering algorithm is proposed to compute the weights of edges. A global optimal action sequence matching method based on dynamic programming is proposed, and the distance between the weighted graph and the feature sequence is calculated. This algorithm can calculate the similarity between the weighted graph and the action sequence with arbitrary length. This algorithm does not need to pre-segment the sequence and solves the problem of the temporal dynamics. Compared with using vector space to represent actions, the temporal correlation between the clustering centers is described by weighted graphs. In the testing stage, some experiments testing the effects of different clustering methods and different vertices of graphs are performed and have proved that our algorithm is robust.