Tomato Picking Robot Fruit Recognition And Positioning Technology Research

With the continuous promotion of agricultural intelligence,the application of artificial intelligence technology in agriculture,represented by deep learning and robot,has been widely studied.To address the shortcomings of the current tomato picking robot vision system,this paper proposes a tomato identification and localization algorithm based on YOLOv5 s and point cloud data,optimizes the picking process,and tests the performance of the tomato picking robot through experiments.The main research contents are as follows:First,the design of the tomato picking robot vision system was carried out.The robot was built using ROS(Robot Operating System).The hardware adopted the “eye in hand” scheme,modeled as a whole by URDF according to the hardware connection relationship of each part,and the hand-eye calibration was performed.Next,the original YOLOv5 s network structure and the traditional NMS(nonmaximum suppression)method were improved.Cross-layer connections were added to the Neck(neck network)of YOLOv5 s for multi-scale feature fusion.In addition,the detection frames that exceed the overlap threshold were processed using the decay function,which helps to solve the problem of high detection missing rate caused by mutual occlusion of fruits.Then,the position information of the fruit is obtained by combining the target-based detection results with the depth image and coordinate transformation.In addition,this study proposes a method to judge whether the picking is successful or not based on vision system,and terminates the process of unsuccessful picking in time to form a closed-loop feedback,which improves the picking efficiency.Finally,the robot platform was built for carrying out experiments.The experimental results show that the improvement of YOLOv5 s in this study is effective,and its recognition accuracy reaches above 97.62%,2.38% higher than that of YOLOv5 s,and at the same time,its missed detection rate is reduced by 4.76%.The picking experimental data show that the mean absolute error of the fruit grasping position is no more than 3mm,which can meet the requirements of accurate picking.In summary,taking tomatoes as the research object,this paper studies the vision system of tomato picking robot and the picking process based on deep learning and point cloud data,which provides some references for the design and research of tomato picking robot.