Research And Implementation Of Human-machine Cooperation And Interaction Technology Based On Visual Grabbing

With the concept of Made in China 2025 put forward,intelligent manufacturing technology will become the hot research direction of industry in the next decade.As a representative high level manufacturing industry,industrial manipulator has more extensive application scenarios.Manipulator based on teaching device programming has low level of automation and intelligence,but the combination of the manipulator and vision makes the manipulator more flexible.Although vision-integrated robots can accomplish many excellent tasks,some special tasks require human participation.Therefore,human-machine cooperation can be used to accomplish tasks.In order to improve the interaction ability between human and manipulator and the intelligent level of manipulator.This thesis studies human-computer cooperation and interaction technology based on visual capture.Using visual sensors,the manipulator completes the planned cooperative task by understanding different gesture meanings.Firstly,the studied visual grab system is the type of eye in hand.For this vision system,it is necessary to obtain accurate hand-eye relationship.The traditional hand-eye calibration method has a unique solution theoretically,but due to many constraints,it is easy to lead to the instability and non-uniqueness of the solution.Through a lot of experiments,the traditional calibration method is improved.The method named a least square based on motion constraints is proposed,and the constraints and specific algorithms are given.Processing experimental data proves that the solution obtained by the improved algorithm is convergent and unique.Therefore,it has certain theoretical significance.Secondly,in order to realize the function of visual capture,the monocular vision positioning theory is studied.Using the principle of "one eye,two positions" to locate single target point.Two frames of images are acquired in different positions and postures.The image is processed by visual processing library to obtain the feature points of the workpiece.Combining with the results of hand-eye calibration,the kinematics theory and coordinate transformation theory of the manipulator are used to determine the three-dimensional position of the target point.Finally,aiming at the problem of man-machine cooperation,two kinds of human gestures are given in the experiment,including fist and palm.Using visual processing library,different image processing processes of human gesture recognition are designed.When the fist is recognized,the trigger arm completes the task of grasping the workpiece.When the palm is recognized,the trigger arm completes the cooperative task of grasping the work placed in the palm.By building a visual grabbing platform,using manipulator library function,OpenCV visual processing library and so on,an interface is developed to realize the automatic calibration of hand-eye system and human-machine cooperation and interaction functions.The experiments of vision-guided positioning and grabbing,and human-machine cooperation are implemented.Aiming at the experiment of vision-guided positioning and grasping,the positioning errors of checkerboard corners and the grasping position errors of workpiece are analyzed.The average positioning error is less than 2 mm,which verifies the feasibility of the improved hand-eye calibration algorithm and the effectiveness of visual grasping.Aiming at the human-machine cooperation experiment,different human-machine cooperation tasks are planned and completed by understanding different human gestures,which proves the feasibility of human-machine collaboration and interaction technology and has certain engineering application value.