The Design Of Tele-operation Manipulator Controlled By Hand Gesture

Tele-operation manipulator can complete complex task in extreme environmentsinstead of human being, to improve working efficiency and ensure personal safety.The traditional manipulator can be controlled by a mechanical rocker or computerkeyboard, and be placed in the cable network, which greatly limits the flexibility ofthe manipulator.In this paper, tele-operation manipulator is combined with gesture detectionbased on computer vision. This system uses digital camera to collect the image ofgesture and converts gesture command into control command of the manipulatorservo. Furthermore, control command will be sent through a wireless network to themanipulator control unit, to realize the remote control of the manipulator.The main work of this paper is as follows:(1) Combine double criterion for gesture segmentation. Firstly, image should bepreprocessed and converted to YCbCr color space. For static hand gesture, we use thesegmentation based on skin color with an improved Otsu method. For dynamic handgesture, we combine the method of frame difference and background difference toseparate dynamic hand gesture from static background. Finally, we establish a modelbased on the speed of the dynamic gesture to merge the skin color image and dynamicgesture image.(2) Detect hand gesture by the count of fingertips. Firstly, we find the contour ofgesture from the binary image, then find all fingertips using the method of maximumcurvature, and find the center of palm by separate arm from palm. Finally, afterchecking gesture, we get correct hand gestures.(3) Convert gesture into control command. By extracting dynamic gesture trackfrom a sequence of video frames, we convert hand gesture into the control commandof the manipulator.(4) Control method of tele-operation manipulator. We take the Atmega16asMCU to control six-degree-of-freedom manipulator by control its servo. Byestablishing the wireless network via Bluetooth and serial communication, thissystem achieves remote control of manipulator by hand gesture. (5) Establish a human-computer interaction software. Through this software,user can open camera, initialize wireless network, check detection of hand gesture,and control manipulator by performing a hand gesture or sending command.