Real-time Action Recognition In Interaction Application

Human action recognition is a significant research point in Computer Vision and has been widely used in new way of human-computer interaction, intelligence system and other fields. Therefore, it is of great value to recognize human action accurately in real-time. In this paper, we learn a real-time human action recognition method and an application in human-computer.(1) A local feature is proposed to describe the local joints structure information of human. Then the Histograms of 3D Joints(HOJ3D) are combined to represent the human posture. Human joints locations are acquired by depth images from RGB-D sensor, and are processed by filter in time to reduce influence caused by jitter. After this procedure, the accuracy of joints location is improved and we extract two types of feature based on joints data as the final posture description. Linear Discriminant Analysis(LDA) is utilized to reduce the feature dimension. The experiments show that Local Joints Structure feature is able to solve the problem which HOJ3 D ignores, and accuracy given by combining HOJ3 D and Local Joints Structure is better than the result by only using HOJ3 D.(2) A real-time human action recognition method based on Hidden Markov Model(HMM) is proposed. The feature proposed in(1) is firstly calculated and key poses are generated using K-means clustering. Then an action is changed into a sequence of key poses and they are passed into those trained HMMs as observation variables. For each action, we train a HMM using Baum-Welch algorithm and all HMMs are used to recognize an unknown action. Experiments demonstrate that, the proposed approach achieves better performances and higher efficiency.(3) A flexible and scalable human action recognition based interactive application is designed and realized. The system includes the perception module, action segmentation module, recognition module, interactive module and model driven module. In the perception module, the RGB-D sensor is used to get RGB-D image and human joints data. In the action segmentation module, the complete action data are extracted by deciding the start and end point. Then they are passed into the recognition module. In the recognition module, two types of features are extracted and action is recognized. In the interactive module, the response action data are acquired according to recognition results. In the model driven module, human avatars are driven by action data in the following process and interact with users. As a result, it achieves an interesting interactive application, and also has good interactive experience.