Dual Manipulator Cooperative Control Based On Binocular Vision

Robot technology has developed rapidly since the robot came into being more than half a century ago.Single-arm robots are limited and can not meet the needs of today's production and life.Dual-arm robots have high flexibility and strong load capacity.They can play an important role in industry,service industry,medical and disaster relief,and other fields.They have attracted extensive attention of researchers.Cooperative control of dual manipulators is difficult,and its ability determines whether the robot can accomplish the task or not.Based on this,this paper will study the cooperation of dual-arm robot from the perspective of dual-arm cooperative control based on binocular vision,aiming at completing the cooperation task of dual-arm robot.The main contents of this paper are as follows:Firstly,two Kinova Mico~2 manipulators and a ZED binocular depth camera are used to build the hardware platform of the dual-manipulator cooperation system.In the ROS system,the URDF model of dual manipulators is established,and the data of URDF model and the angle information of each joint are obtained.Then the motion state of robot is obtained and displayed and simulated in Rviz.Secondly,based on OpenCV computer vision library,a multi-target recognition and location method is designed.Based on the HSV color space,the contour centers of the objects with different colors are extracted to obtain the two-dimensional coordinate position of the center point,and the three-dimensional coordinate position is obtained by combining the depth information of the center point in the depth image.Then the deflection angle of the target object is obtained by using the two-dimensional enclosing rectangle.This method is simple to implement and can obtain the geometric characteristics of the target object accurately and effectively.The method has good robustness.In addition,the transformation relationship between the camera coordinate system and the manipulator coordinate system is released by TF in ROS,so that the pose transformation of the target object from the camera coordinate system to the manipulator coordinate system is completed.Finally,the task of pouring water is accomplished with two manipulators.On the basis of the position of the target object in the coordinate system of the manipulator,the posture of the end of the manipulator is obtained according to the azimuth angle,polar angle and end rotation angle of the target object relative to the root coordinate system of the manipulator,and the angle information of each joint of the manipulator is obtained through inverse kinematics solution.Based on the cubic interpolation polynomial,the trajectories of the two manipulators are planned respectively,and the two manipulators are driven to grasp different objects in a gradual manner and reach the designated position in space,and the end joint of the manipulator holding the water bottle are rotated to complete the action of pouring water.