Research On Recognition And Location Method Of Citrus Picking Robot Based On 3D Vision

Citrus picking is the most time-consuming and laborious part of the whole citrus production process.It is of great significance to develop an intelligent citrus picking robot to improve operation efficiency and reduce picking cost.The recognition and positioning of citrus fruits by visual system is the most critical part of the whole citrus picking robot,and its performance will directly affect the success rate of fruit picking.However,in the complex natural environment,problems such as natural light changes,branch leaf occlusion,and fruit overlapping occlusion often lead to problems such as low recognition accuracy or low positioning accuracy.Therefore,it is of great practical significance to study a set of visual recognition and positioning system with good real-time performance,high recognition accuracy and high positioning accuracy for citrus fruits growing in complex environments.In this paper,citrus fruits grown in natural environment are taken as the research object,and the recognition and location of citrus fruits are studied based on convolutional neural network and 3D visual positioning technology.The main research contents and results are as follows:1)A citrus picking robot vision system was constructed and citrus samples were collected.By analyzing the existing 3D vision technology and combining the advantages and disadvantages of different technologies,the Real Sense D435 i depth camera based on stereo active vision is selected to build the vision system.Based on the comprehensive analysis of the fruit distribution range on the tree,the ideal working distance of the camera and the working radius of the manipulator,the optimal working height of the camera is about1.15 m,and the optimal working distance is 1.04 to 1.3 m.In order to make the robot adapt to different hilly terrains and natural orchard scenes,citrus sample images were collected at different shooting angles and different light intensities,and a data set suitable for autonomous picking of citrus in natural orchards was constructed.2)A citrus fruit target recognition model based on improved YOLOX-s was constructed.Considering the requirements of citrus picking robot for recognition accuracy and real-time performance,a deep learning model based on YOLOX-s is selected.Aiming at the problem of cluster and branch occlusion of citrus fruits,detailed classification rules are formulated for fruits with different growth postures and occlusion conditions on the tree from the perspective of picking robots.A lightweight ECA attention mechanism is introduced and Focal loss is used as a confidence loss to improve the model.The experimental results show that the improved model can effectively reduce the false recognition.The average recognition accuracy of the four types of targets is 92.95 %,which is 0.31 % higher than that before the improvement.The improved model has the highest recognition accuracy for directly picking fruits,reaching 99.29 %,and the recognition accuracy for strategic picking,non-picking and branch targets is 90.66 %,96.03 % and 85.83 %,respectively.3)Fruit three-dimensional space positioning based on RGB-D depth camera.Using the recognition results of the improved YOLOX-s model,the focus is on researching the threedimensional spatial positioning method for directly picking fruit targets.The fruit positioning experiment was designed and completed.The average positioning error of citrus fruit in the Z-axis direction was 3.4 mm.According to the actual scene,the hand-eye calibration experiment of citrus picking robot is completed.4)Visual system integration and picking experiment of citrus picking robot.Using PySide6 to complete the development and function realization of Graphical User Interface(GUI).The three-dimensional positioning and picking experiment of citrus is carried out in the indoor scene.In the Cartesian space coordinate system of the manipulator,the average positioning errors of the picking robot system in the X-axis and Y-axis directions are 2.01 mm and 1.7 mm,respectively.For fruits that can be directly picked,they can be correctly identified and picked,and the success rate of picking fruits that can be directly picked in laboratory scenarios has reached 100%.