Research On Fruit Recognition And Positioning And Post Harvest Automatic Grading Technology Of Picking Robot Based On Deep Learning

Fruit picking robot and Fruit Sorter can realize the automation of picking and sorting,and can well solve the problems of insufficient labor force,high labor cost and low efficiency,poor competitiveness and so on.The focus and difficulty of the research and development of picking robots and sorters lies in the vision system.Their working efficiency and stability depend on the speed and accuracy of fruit recognition.Therefore,the research on the vision system that can accurately detect and locate the fruits on the tree in the complex environment of the orchard and classify the fruits with different appearance quality in the environment of the sorting production line is very important for realizing automatic picking Yield estimation and automatic sorting have important research value and practical significance.Taking apple as the research object,based on deep learning and binocular vision technology,this paper studies the automatic grading technology of postharvest fruit and the fruit detection and positioning technology of picking robot.The main research work and results are as follows:1.Study on fruit detection methods.In order to enable the picking robot to quickly and accurately recognize the fruits with different maturity in the orchard complex environment such as different lighting,overlapping occlusion and large field of view,a fruit recognition method based on improved YOLOv3 is proposed in this study.Firstly,the residual module in DarkNet53 network is combined with CSPNet(cross stage parallel network)to maintain the detection accuracy and reduce the amount of network calculation;Secondly,SPP(spatial pyramid pooling)module is added to the detection network of the original YOLOv3 model to fuse the global and local features of fruit and improve the recall rate of minimal fruit targets;At the same time,Soft NMS(soft non maximum suppression)algorithm is used to replace the traditional NMS(non maximum suppression)algorithm to enhance the recognition ability of overlapping occluded fruits;Finally,the joint loss function based on Focal Loss and CIoU Loss is used to optimize the model and improve the recognition accuracy.Taking apple as an example,the test results show that the MAP(mean average precision)value of the improved model trained by the data set reaches 96.3%,which is 3.8 percentage points higher than that of the original model;The F1 value reached 91.8%,which was 3.8 percentage points higher than the original model;The average detection speed under GPU is 27.8 frames/s,which is 5.6 frames/s higher than the original model.Compared with several current advanced detection methods such as Faster RCNN and RetinaNet,and the comparative test results under different numbers and different illumination show that this method has excellent detection accuracy,good robustness and real-time performance,and has important practical value for solving the problem of accurate fruit recognition in complex environment.2.Research on three-dimensional positioning method of fruit.After the two-dimensional coordinates of the fruit picking center point are obtained by the target detection algorithm,in order to further obtain the depth information of the fruit picking point,the binocular stereo vision technology is used,the Zhang calibration method is used to calibrate the binocular camera,determine its mathematical model,and stereo correct the collected image,The corrected left eye image and right eye image are stereo matched by sgbm stereo matching algorithm,and finally the three-dimensional coordinates of the fruit picking center are obtained through parallax calculation.The experimental results show that the average positioning error in the depth direction of the three-dimensional coordinates obtained by the vision system is±15mm,which meets the actual picking requirements.3.Study on fruit classification methods.In order to realize the fast and accurate appearance quality classification of picked fruits,and cooperate with the sorting production line to complete the large-scale centralized sorting of fruits,a fruit classification method based on improved ResNet is proposed in this study.Firstly,the residual module in ResNet network is combined with the dual channel SE(Squeeze-and-Excitation)module to enhance the effective channel features,suppress the inefficient or invalid channel features,and improve the expression ability of the feature map,so as to improve the recognition accuracy;Secondly,the Inception module is added to the original ResNet model to fuse the characteristics of different scales of fruit,so as to enhance the recognition ability of small defects;Finally,four kinds of fruit images with different appearance quality were enhanced,and the model was initialized by transfer learning method.Taking apple as an example,the experimental results show that the accuracy of the improved model trained by the data set is 99.7%,which is higher than 98.5% of the original model;The precision rate is99.7%,which is higher than 98.3% of the original model;The recall rate reached99.7%,higher than 98.7% of the original model;The average detection speed under GPU is 32.3 frame/s,which is basically the same as the original model.Compared with several advanced classification methods such as GoogleNet and MobileNet,and compared with different improved models,the results show that the method has excellent classification accuracy,good robustness and real-time performance,and has important practical value for solving the problem of accurate classification of fruit appearance quality.4.Software design and implementation of fruit detection,location and classification system.Based on PyQt5 and Qt Designer,the apple detection and positioning software of apple picking robot in complex orchard environment and the apple appearance quality classification software in automatic sorting production line environment are designed to realize accurate detection,positioning and appearance quality classification of apples.