Research On Obstacle Recognition And Location Method Of Citrus Picking Robot

China is a big agricultural country of citrus production,and the citrus industry is very developed.However,in the citrus harvesting work,there is no automation and the operation efficiency is not high,which has become a problem to be solved in the citrus industry chain.At present,the first generation of Citrus picking robot developed by our research group can complete simple picking work.The robot uses traditional machine vision technology as the module of fruit recognition,and obtains the three-dimensional information of the target object by binocular stereo matching technology to guide the manipulator to complete the picking operation.However,the recognition module has some limitations on multi-target object recognition in natural environment,which limits the working environment of the picking robot.Therefore,the upgrade of multi-objective object recognition system,especially the obstacle recognition in the picking process,has important practical significance and scientific research value for enhancing the application of picking robots in a variety of scenarios.In this paper,citrus trees in mature season are taken as the research object.The recognition module of picking robot is studied to improve the ability of picking robot to recognize obstacles and obtain the three-dimensional coordinates of obstacles.The main content of this paper is as follows:(1).Select recognition algorithm of recognition system.Discusses the common detection models of convolutional neural network in object recognition field,from the RCNN model of the beginning of object detection to the YOLO model of greatly improving recognition speed,and then to the Mask RCNN algorithm,each model has its own advantages for different tasks.The research of obstacle recognition and location methods involved in this research group is more suitable to adopt Mask RCNN algorithm which is more accurate in object segmentation.(2).Training data set making rules.In order to ensure the high similarity between the data environment of the data set and the real picking environment,shooting distance and shooting angle are proposed.The natural distribution of citrus fruits in ripening season was studied.Six classification rules were proposed for identifying target species,namely NM,BO,LO,SO,OL and MB.These six kinds of targets cover all kinds of distribution of citrus fruits in natural environment,and they are representative to some extent.Rotation,salt and pepper noise and horizontal flip data enhancement technology are used to expand the data set to prevent the training model from over-fitting.According to the task requirement of this subject,the rotation angle is set to 0-25.A grid marking method is proposed for irregular trunks to improve the recognition accuracy of the model.(3).Recognition of obstacles.The ResNet50 backbone structure of Mask RCNN is studied.In order to improve the recognition accuracy,ResNet152 with deeper network layers is used as Mask RCNN backbone network.Using transfer learning training,the experimental results show that the improved MaskRCNN-152 recognition and detection model has a comprehensive recognition accuracy rate of 86.83%,which is 12% higher than that before the improvement.The grid trunk marking method improves the accuracy of trunk recognition by 37.83%.The recognition model achieves 85% comprehensive recognition rate under varying illumination conditions,and has good robustness.(4).3-D spatial location of obstacles.The Kinect V2 camera is calibrated and get the camera's internal and external parameters.The three-dimensional coordinates of the center point of the object are calculated by Kinect V2 three-dimensional coordinate measurement model,and the three-dimensional coordinates of the center point of the object in the coordinate system of the manipulator are calculated by hand-eye calibration.The experimental results show that the positioning error of Kinect V2 three-dimensional measurement model is about 3 mm.In the natural environment,the success rate of harvesting fruits and obstacle avoidance of harvesting robots reached 85% and 83% respectively.