Research On Robotic Manipulation System Based On Visual Gesture Recognition

With the rapid development of robot technology,the robot has been widely used in production,life and other fields.The interaction between human and robot becomes more and more frequent.But at present,the operation method of robot is still monotonous and cumbersome,and the interaction between human and robot is not intuitive,natural and inefficient.Therefore,the research of natural gesture interactive system based on vision not only has a certain academic value,but also has considerable practical application prospect.In this paper,we analyzed the domestic and foreign research materials and results related to gesture recognition and robot control system,and selected the Leap Motion sensor produced by the Leap company to get hand information.For the gesture recognition accuracy reduction phenomenon which is due to visual interference or inevitable occlusion when a single Leap Motion sensor is working,we has designed and realized a hand gesture control system of manipulator based on multiple Leap Motion sensors.The system adopts multiple sensors in different directions detecting hand gestures and completing gesture recognition,and realizes the non-contact interaction of controlling the manipulator by natural gestures.This system consists of client(gesture detection module),information server(data processing module),mechanical manipulator controller and manipulator,this paper mainly studies and discusses the gesture acquisition module and data processing module.The first part of this dissertation introduces the working principle of the manipulator based on gesture recognition.After the study and analysis of the Leap Motion sensor,the paper determines the location layout of multiple sensors and then a gesture data acquisition module based on multiple sensors is realized.The main function of data processing module in this system is to analyze and process the gesture data acquired from multiple sensors.In order to response to changes in the hand motion state more timely and obtain a stable,accurate hand position data,the paper puts forward a improved adaptive Kalman filtering based on the current statistical model to tracking the hand.The joint angle which is obtained by calculating the vector of finger phalanges,becomes more stable and smooth by a improved weighted moving average filtering algorithm based on mean value.When the hand gesture recognition is required,the feature vectors such as finger joint angles for gesture classification are put into support vector machine(SVM)for hand gesture classification and gesture recognition accuracy is up to 98.5%.The data processing module can sent the position increment,rotation angle increment or gesture commands to the robot controller in real time,then the manipulator controller will get the joint movement data of the manipulator to drive each servo motor through the internal kinematics inverse algorithm.In the final experiment,the operator can control the manipulator by changing the spatial position and posture of the hand,which verifying the availability of the system and the effectiveness of the data processing algorithm.