Research Of Robot Arm Control Based On Visual Hand Gesture Recognition

In robot arm application field, the difficulties come from complex and changeable environment. The non-traditional environments(radiation, deep seas, remote, etc.) are not appropriate for human work directly, and the household environment needs real-time and convenient interaction. Visual hand gesture control technology is intuitive and flexible, and the difficult which robot arm cannot handle autonomously can be solved in complex environment. The study of intelligent robot arm system based on visual hand gesture has important theoretical and practical significance.The main meaning of visual hand gesture is dynamic gesture trajectory and static gesture shape. In this thesis, dynamic gesture trajectory and static gesture shapes are used to real-time control robot arm terminal trajectory and end-effector respectively, and the intelligent Human-computer Interaction system is designed. In order to maximize real-time ability and stability of the robot arm control, the visual hand gesture processing algorithm and system design are studied deeply and implemented.In marker-less hand gesture segmentation and three-dimensional trajectory extraction, the Kinect camera and depth threshold algorithm are selected in hand gesture detection. For the problem of wrist interference, the mark-less hand gesture is segmented accurately by proposing eliminate wrist algorithm based on dynamic threshold. After that, by combining depth information, the real-time three-dimensional trajectory is extracted. For the problem of redundant trajectory and abnormal gestures, the time slice average and sphere threshold limited algorithm are proposed.For the problem of lack robustness and real-time ability, in hand gesture feature extraction and recognition, the Kernel Density Estimation Sequence of Fingertip Angle Set(KDES-FAS) is extracted by defining fingertip angle set and using kernel density estimation theory. Then the hand gesture shapes are recognized in real-time and robustly by selecting template matching algorithm.After the completion of hand gesture trajectory extraction and recognition, the robot arm kinematics and simulation of system design are studied. The robot arm coordinated hand gesture control simulation is completed in MATLAB environment by analyzing 6R robot arm kinematics and system module design. Finally, implementation of visual hand gesture control robot arm. On 6R industrial robot platform, dynamic gesture trajectory control robot arm terminal trajectory is implemented steadily, and the end-effector control error rate is decreased to 2.33%. The experiments confirmed that the visual hand gesture processing algorithm and control solution is feasible and superior.