Implementation Of Human Action Recognition System Based On Kinect

Study on the recognition of human action is the foundation of action understanding, and it is also an important prerequisite for the friendly human-computer interaction, furthermore, it has important research significance and wide application background. With the continuous development of social intelligence and informatization, more and more scholars dedicated to the study of how to realize the human action recognition accurately and robustly in order to better understand human action intention, so as to make robots and other intelligent devices to provide more convenient service for the people.At first, the scholars studied a variety of advanced image-processing algorithms for two-dimensional video images to recognize the action, but this method can not avoid the interference of ambient noise, such as color, texture, illumination, occlusion and so on. Along with the development of motion capture technology, it has come true to capture a large number of 3D spatial data sets based on three-dimensional space. Kinect is a 3D somatosensory camera launched by Microsoft Corp in 2010, it can not only capture the RGB color image like ordinary camera but also measure the depth of the scene. Not only that, Kinect can also provide three-dimensional coordinates of some important joints. In view of the 3D motion data set can better replicate the motion details, and it can record the trajectory more accurately. In this paper, we use Kinect as the external equipment to capture 3D coordinate information of 20 joints of human body, finally we realize the recognition of human action in a small range. On the basis of analysis in human action representation methods and action recognition algorithms, we carry out the research on methods of human action recognition treating the human skeleton model as the research object. In addition, in order to use the semantic information of action in the video effectively in the next step, we detect the human hand from the video. The main work of this paper are as follows:(1) We used Kinect to capture the depth image, color image and skeleton image, and we mapped the skeleton image to the color image by coordinate transformation, so that the developers can observe the positioning precise of human joints more intuitively, furth ermore, it is convenient for developers to assess the tracking state of the joints so as to ensure the quality of skeleton data.(2) One significant difference of Kinect and ordinary camera is that Kinect can provide the location information of 20 joints, and it can accurately give the 3D coordinates of our joints, on this basis, the existing joints data were normalized. Finally, we used normalized data as the low-level features for action recognition.(3) The distribution of color camera and depth camera is in different positions of the Kinect, therefore pixels in the two images may not always line up exactly. Hence, we calibrate the RGB camera and depth camera of the Kinect.(4) In order to make full use of the semantic information in the environment and avoid the subjectivity and incomplete duo to feature sets defined by human, we can extract operating objects characteristics, this semantic characteristics is very effective for action recognition in a specific scenario. Since most action containing operating objects are performed by the hand, in order to identify the operation objects, we must first detect the hand. We used the depth and skin color information to detect and locate hand’s position in this system.(5) After the extraction of the low-level features, we selected BP neural network as the classifier to train the characteristics data, according to the classification results, we built the hierarchical BP neural network improved. Owing to the execution of second layer network depended on the classification results of first layer in the network, so this classification method is called hierarchical BP neural network of results driven. We save the weights trained by BP neural network and call them in VS. Finally, we obtain the action recognition results by the forward calculation formula of the BP neural network.We described the function requirement analysis of the system in detail. This system is divided into three parts, data acquisition module, feature extraction module and action recognition module. The main function of data acquisition module is to use Kinect to capture the underlying data stream. The main function of the feature extraction module is to extract action feature vectors based on the joints coordinate data and detect the human hand. The main function of action recognition module is to train the extracted feature data and output the recognition results. According to the three modules of the system, we give out the detailed design of the system. The experimental results show that the effectiveness of the classification algorithm designed in this paper, the accuracy of the action recognition system can reach the effect which we expected.