Research On Accurate Identification And Localization Of Tomato Target And Harvesting Strategy

The labor force involved in agricultural production continues to decline as China’s aging population and urbanization accelerate,and agricultural production faces a workforce shortage and fast rising labor costs.Tomato production,in particular,is steadily changing from small-scale to large-scale greenhouse farming,and labor conflicts are becoming more visible.As a result,it’s critical to encourage a shift from manual harvesting to robotic harvesting.The research object in this paper is cherry tomatoes,and it forces on the technical problems of target identification,grasping pose calculation,and harvesting strategy of picking robots in an unstructured environment.The following are the main research contents of this paper.(1)Design and construction of a vision servo system for a tomato harvesting robot.According to the needs of cherry tomato growing agronomy in the greenhouse and robot intelligent harvesting operation space,the "Eye-in-hand" vision system was built.First,the hand-eye calibration method and the control principle of the harvesting robot’s vision servo system are explained,using the six-degree-of-freedom collaborative robot arm as an example.Then,the built vision servo system’s hand-eye calibration is performed,and the hand-eye transformation matrix is solved.Finally,the hand-eye vision system’s positioning accuracy was assessed,and the average positioning error of the camera at a distance of 600 mm from the target is 2.91 mm measured inside.(2)The method of locating the tomato peduncles,as well as computing the pose of the tomato clusters is investigated for harvesting the entire tomato cluster.To address the problem of large positioning error and false shearing,an accurate method for locating the cutting point of the tomato peduncle based on a combination of close and distance view is proposed.First,YOLOv4 object detector is used to identify and locate the entire cluster of tomatoes.After that,the camera approaches the target tomato cluster,and YOLACT++instance segmentation algorithm is performed to segments the pixels of the tomato cluster and the peduncle in the close-up image.Then,curve fitting is performed on the peduncle pixels to locate the shearing points,and the success rate of identifying the shearing points of tomato peduncles achieved in the field test is 93.4%.Finally,a mathematical geometric model is established based on the cutting point and the feature points to calculate the tomato pose.Accurate identification of peduncle cutting point and calculation of peduncle pose can provide the basis for accurate trajectory planning of robotic arm movements.(3)Research on identification,grasping pose estimation and harvesting strategy for autonomous harvesting tomato fruits.In the process of tomato fruit harvesting,in order to reduce collisions between the end-effector and the surrounding obstacles,object identification and localization,harvesting sequence and pose calculation methods are proposed.First,the YOLO object detection method is used to detect and locate tomato fruits and tomato clusters in the image.Then,the tomato picking order is optimized based on cluster determination and nearest distance method within tomato bunches.Finally,an end-effector grasping pose adjustment method is proposed to make the end-effector approaching the target tomato in a reasonable pose.In the field test,the results show that the tomato fruit recognition success rate is 97.3%,and the tomato harvesting success rate in the whole machine test reaches 72.1%by using the optimized harvesting strategy and end-effector grasping pose estimation method.(4)Optimal design and experimental research of the tomato harvesting robot in the production greenhouse.First,the hand-eye vision system,vision algorithms and harvesting robot hardware are integrated to build a tomato harvesting robot prototype.Then,we develop the control system of the harvesting robot and multiple rounds of tests are conducted to verify the effectiveness of each vision algorithm module in a production greenhouse.Finally,the overall harvesting success rate of the proposed tomato harvesting robot reaches 72.1%,and the average picking time of each tomato is 14.6 seconds.