The Design And Implementation Of Natural Human-Robot Interaction System Based On Kinect Sensor

Human-robot interaction technology has been developing for many years, but the interaction between human and machine has been unable to get rid of the external devices, such as the keyboard, mouse and remote controller which has restricted the promotion of interaction experience. In recent years, with the fast development of artificial intelligence, pattern recognition and machine learning, human express a increasingly strong desire for the natural interaction. The so-called natural interaction is to make the machine to observe the action, listen to the voice of the human and eventually understand human’s instructions that makes communication more natural. In this paper, we study natural human-robot interaction system based on Kinect sensor, the main research content is as follows:Firstly, this paper introduces the hardware of Kinect sensor and related core technologies, including structured light coding, skeleton tracking and speech recognition. Then realizing gesture recognition and hand shape recognition with color image data, depth image data and skeleton data provided by Kinect sensor. And this paper emphatically introduces the gesture recognition based on 3D coordinates of skeletons, analyzes the principle of static hand shape recognition based on Hu moment and MLP neural network, also compares the effect of the two methods through experiments. For dynamic gesture recognition, we propose a recognition method based on hand position, angle and speed, and verify its feasibility. By calling on the speech recognition API of Kinect for Windows SDK, voice command recognition is realized. Finally, we discuss the design and implementation of hardware and software for the system in detail, and give the necessary circuit schematics and software flow chart.For all functions of the system, this paper have carried out the necessary experiments, and the results show that the system can achieve given interactive functions and has good robustness, so we has achieved expected effect of research.