Research On Kinect-Based Gesture Recognition And Robot Control Technology

With the development of the robot control technology, interactions between humans and robots are becoming increasingly common. The use of human actions and gestures to control the robot can replace complex and cumbersome program operations, and humans can manipulate and interact with the robots easily and conveniently. Human motion and gesture recognition based on video is an indispensable key technology in the new generation of human-computer interaction technologies. As a revolutionary product, the Kinect device can obtain color images, depth images as well as images of the human skeleton and provide a new way for human-computer interaction. It can capture, track and decrypt the human motions, gestures and sounds. In this thesis, research on static gesture recognition and dynamic gesture recognition are carried out by using Kinect. Based on this, gestures are recognized and encoded as control commands for robots, so non-contact interactions between humans and robots can be realized.In this thesis a set of static gestures and dynamic gestures are defined to control a robot’s actions. For static gesture recognition, the palm portion is segmented from the whole image using the depth information provided by the Kinect device, and a method to detect the finger contour based on the dual layer depth information and the human skeleton information is proposed, and it can detect the contour of both straight fingers and bent fingers. Based on this, by adopting the K curvature algorithm for determining fingertips, static gestures can be recognised. For dynamic gesture recognition, the Dynamic Time Warping (DTW) algorithm is adopted, and the static gesture recognition is integrated with the dynamic gesture recognition to improve the accuracy of gesture recognition and the amount of composite gestures, so the corresponding instructions for robots has been increased greatly. After achieving the static and dynamic gesture recognition, valid gestures are encoded as instructions to be sent to the robot control system by wireless communications, and then the robot can respond to human gestures and interact with humans.The static and dynamic gesture recognition methods based on Kinect proposed in this thesis can effectively recognize human gestures, and by using the image depth information of human gestures, the influence of illumination on human bodies can be greatly reduced, and the stability of gesture recognition can be improved. By integrating gesture recognition with the robot control system, robots can interact with the human gestures, and it has good application value in the future.