Research On Hand-eye Coordination Technology Of Robot Picking Grapes

III Grape is one of the most widely distributed fruit tree species in China because of its sweet taste,rich nutritional value and obvious role in food and health care.At home and abroad,grape harvest and picking mainly by hand are directly affected by the continuous decrease of rural labor population in towns and villages.Meanwhile,manual picking is inefficient and the workload is large.However,there is still a lack of complete and systematic solutions to the research of grape picking robot in China for the specific cultivation mode of grape picking operations.Therefore,the research content of hand-eye coordination technology is of great scientific significance for the realization of grape picking robot's completely autonomous operation.The main research contents of this paper are as follows:Firstly,the cultivation mode of the hedgerow edible grape plantation and the overall structure design scheme of the picking robot are introduced,and a visual recognition and positioning system of the far-close-range combination is constructed.The binocular vision system is applied to the image collection of the orchard,and the coordinated motion strategy of the multi-image combination is developed.The H component in the HSV color space is subjected to median filtering,automatic segmentation of qualifying threshold,morphological operation,executable region marking,extraction of the target centroid,etc.,to complete the identification and feature extraction of the grape.The technique of identification and localization of near-view fruit stems was studied.Based on the vertical suspension growth characteristics of grapes,a calculation model for cutting points of fruit stems was established.Edge detection and cumulative probability Hough transform were carried out in the selected region of interest of the fruit stalk,and the straight line segment of the fruit stalk edge was screened based on the angle constraint and the minimum straight line distance from the grape centroid point to the fruit stem,and the midpoint of the line segment was extracted as the fruit stem cutting point.Calculate the end pose based on the near-field positioning information and the ear equatorial circle depth point cloud data.Then,the LabVIEW-based camera calibration,stereo matching and 3D recovery process are studied,and the grape image coordinates acquired by the vision system are converted into spatial three-dimensional coordinates in the binocular camera coordinate system.The robot coordinate system is established.On this basis,the process of coordinate transformation from the visual coordinate system to the mechanical arm coordinate system is analyzed.The hand-eye calibration is used to solve the homogeneous transformation matrix between the camera flatform coordinate system and the robot arm coordinate system,the real camera coordinate system and the end effector coordinate system.The inverse kinematic inverse solution equation and the straight path path planning are used to transform the coordinate information into the joint motion control commands of the manipulator.Finally,based on LabVIEW,the development of measurement and control system software was completed,including: binocular vision system program for distant-view grape identification and positioning,camera platform motion control program,close-view fruit stem recognition and positioning program and test system master program.Based on the self-designed grape picking robot test system,the reliability of the close-view fruit stem recognition and localization algorithm was verified,and the success rate of 92% sunny day,82% sunny light and 86% light cloudy day was realized.The distant-close stereo vision system consisting of binocular camera and real-life camera was compared and analyzed.It proved that the distant-close combination of binocular vision system distant-view grape detection and RealSense camera vision system close-view fruit stem identification is reasonable.At the same time,the spatial positioning experiment verified that the two stereo vision systems meet the measurement accuracy requirements of fruit spatial positioning.The hand-eye coordination technology comprehensive test test results of picking grapes show that the test system can successfully complete the complete picking action under the requirements of precise control,and the average time taken in a single time is 53.4s.