Fruit Recognition And Grasping System Based On Deep Learning

The recognition and grabbing of fruits are important components in the fruit sorting process.With the continuous innovation and development of business models,most businesses currently use manual fruit sorting in fields such as fruit e-commerce,unmanned retail,and agricultural product picking.This method has a high cost and is subject to subjective factors from operators,resulting in uneven fruit sorting quality.Therefore,using robots to automatically recognize and grab fruits has gradually become a research hotspot.For fruit grabbing robots,accurately grasping various types and poses of fruits is an important task.To solve the problem of fruit recognition and grasping,this project proposes a fruit pose estimation method based on two finger grippers,which is combined with fruit detection algorithms to achieve fruit recognition and grasping.At the same time,this project designed and developed a fruit recognition and grasping system based on deep learning,and conducted relevant experimental research to verify the effectiveness of this method and system.Firstly,a collaborative robot based recognition and grasping system architecture was established for fruit recognition and grasping scenarios.The hardware structure and characteristics of the UR3 robotic arm and two finger gripper system were introduced in detail;Focused on the construction of a Robotiq wrist camera vision system,analyzed the mathematical principles of RGB camera calibration and robotic arm hand eye calibration,and conducted relevant calibration experiments to determine camera parameters and coordinate conversion relationships between the camera and the robotic arm end effector;At the same time,the communication mechanism of each module in the recognition and grabbing system was selected,and communication between modules and the upper computer was achieved.Secondly,by combining fruit detection with pose estimation,the task of recognizing and grasping fruits is jointly achieved.For the recognition task of fruits,the theory of object detection algorithms was analyzed,and a single stage object detection network with high accuracy and lightweight was selected and constructed.At the same time,a dataset for training the network was produced.A fruit pose estimation method based on two finger grippers is proposed for fruit grasping tasks.The method uses the minimum bounding shape to fit the grasping pose of fruits and verifies its effectiveness in fruit grasping.Finally,a fruit recognition and grabbing system framework was constructed,which includes user layer,algorithm layer,and hardware layer.Completed the construction of the UR3 robotic arm simulation experimental platform in the ROS environment,and achieved simulation and actual control of the UR3 robotic arm by importing the URDF model of the UR3 robotic arm into the ROS;Configure Move It! The mobile group node and OMPL motion planning library use RRT * algorithm to plan the motion trajectory of the manipulator;Building a simulation environment for Gazebo and Rviz robotic arms,achieving control of real UR3 robotic arms in ROS environment;And conduct experiments on the detection and grabbing of fruits separately.The experimental results show that the prediction accuracy of the fruit detection network is 97%,and the average success rate of fruit grabbing is 86%,verifying the feasibility of the fruit recognition grabbing system.