Design Of Vision Servo Control System For Picking Robot Based On Realsense

In view of the existing vision servo-based picking robot system,there are too many image data,need to be processed and the processing progess is time-consuming,so the system has poor real-time performance,low positioning accuracy,etc.Some visual servo systems built with visible light are sensitive to light abnormalities and fruit recognition.Due to the poor accuracy and low recognition efficiency,this paper constructs an RGB-D camera based on the Jaguar mobile robot platform and builds an eye-in-hand picking robot visual servo system.Through the visual sensor,the color and depth information of the target object can be obtained,and the color information,color and depth information fusion methods are respectively used to identify and locate the fruit on the tree.By the coordination between the camera and the mechanical arm,the amount of image processing calculation is dcressed,improving the positioning accuracy of the fruit on the tree.The main research content of this article is as follows:(1)Design of hardware and software architecture of the picking robot visual servo system.This article firstly take a brief introduction about the RGB-D camera of RealSense sr300 and the control objects: Drrobot robot system,multi-level scissors fork designed by team members,360° rotary table,end-effector,etc.And according to the structural size of the mechanical structure and the form of movement of each joint,the kinematics equation of the picking robot is given,as the basis for the control of the picking robot,according to the existing hardware and software interfaces,to build a system framework for the picking robot.(2)The identification and positioning of fruit on the tree,in order to achieve automatic fruit picking on the tree,it is necessary to identify and locate the ripe fruit on the tree firstly.Therefore,in Chapter 3,this paper make a study about the information fusion using depth information,color,and point cloud respectively.The method realizes the identification of fruits on the tree,and obtains the position of the target object relative to the RealSense sr300 through the three-dimensional coordinates of the point cloud,and further calculates the position of the fruit relative to the robot coordinate system according to the kinematics equation of the robot and achieves the target.The position of the fruit relative to the robot is obtained then.(3)The research of hand and eye coordination control strategy between long and close distance.For that the domain coordination control strategy between the hand and eye is insufficient,mainly due to the complexity of image information and the lack of detailed features.This paper proposes and adopts a hand and eye coordination control strategy between long and close distance,which can effectively reduce the amount of data processing,improve the depth detection accuracy,increase the real-time performance of the robot vision servo system greatly.(4)Experimental verification,under the simplified environment of the laboratory,using simulation fruits and self-designed double-panel scales as the main measurement tools,the picking robot prototype was tested for the accuracy error of the picking robot prototype system and the long-range and close-range coordination precision experiment of the robot arm.Real orchard picking operations,experimental results show that the picking robot's movement error can meet the picking operation requirements.In summary,the RGB-D camera RealSense sr300 is used to construct the hand and eye vision system of the picking robot,and its color and depth information are used to realize the self-identification and localization of fruit on the tree.The distance and hand-eye coordination strategy is adopted to reduce the data processing volume and improve the positioning accuracy,experimental results show that the error of the movement accuracy of the system can meet the needs of picking work.The research work in this paper can make a great difference to the practical application of picking robots.