Research On Method Of Tomato Fruit Recognition And Localization Based On Deep Learning

The tomato picking robot can not only effectively reduce labor costs but also greatly improve the efficiency of tomato picking.This paper takes the tomato fruit as the target,adopts the deep learning method to identify the single fruit occlusion,multifruit occlusion,rhizome occlusion and leaf occlusion of the tomato fruit under different light,and then uses the binocular camera to locate the tomato fruit.The function of target detection and positioning is of great significance for the tomato picking robot to improve the picking efficiency.(1)The tomato image dataset is made and preprocessed.Tomato images were collected in Hangzhou,Zhejiang Province in June 2021.The time for collecting sample images is divided into multiple periods in sunny days and cloudy days,ensuring that enough sample images can be collected under different light.The tomatoes in the sample images were manually calibrated using calibration software,and then the data set was expanded to prepare for deep learning training.(2)Aiming at the complex natural growth conditions in the process of tomato picking,such as single fruit occlusion,multi-fruit occlusion,branch and leaf occlusion,and rhizome occlusion under different light conditions,an improved SSD network is proposed.By adding RFB module to SSD network and introducing exclusion loss function and soft-NMS,the detection accuracy of SSD network in complex situations is improved.Experiments show that the improved SSD network can achieve 87.3% and85.6% of the detection accuracy of single-fruit occlusion and multi-fruit occlusion detection for the same type of objects in a sunny environment,which is 7% and 4% higher than the original network,respectively.For the occlusion of different types of objects,the improved SSD network achieves 83.6% and 89% of tomato target detection under leaf occlusion and rhizome occlusion,respectively,which are 6.6% and 5% higher than the original network.In the case of cloudy days and insufficient light,the F1 value of the improved SSD network is 1.3% higher than that of the SSD network in the case of single fruit occlusion.In the case of multi-fruit occlusion,the F1 value of the improved SSD network is 1.2% higher than that of the original network.In the case of leaf occlusion,the improved SSD network has a 4.3% F1 value compared to the original network.promote.In the case of rhizome occlusion,the improved SSD network has a6.3% improvement in F1 value compared to the original network.The three-dimensional spatial positioning method of tomato based on binocular vision and the corresponding positioning experiment research.Firstly,the binocular camera is calibrated and corrected,and then the disparity map is obtained by the improved NCC matching algorithm.Finally,the tomato coordinates are calculated by the triangulation ranging method and analyzed by experiments.(3)A localization experiment was performed on tomato.Firstly,the binocular camera is calibrated and corrected,and then the disparity map of the left and right images is obtained by the improved NCC matching algorithm.Finally,the tomato coordinates are calculated by the triangulation method and analyzed by experiments.Experiments show that the error of the target tomato in the X-axis direction is within4.3mm,the error in the Y-axis direction is within 3.5mm,and the error in the Z-axis direction is within 18 mm.Errors on the tomato will not affect the successful grasp of the tomato.(4)The improved SSD target detection method is combined with the tomato localization method based on binocular vision.The experimental results show that the confidence of the tomato detection frame of the algorithm used in this paper can reach78% in the case of insufficient light,and the localization accuracy can be compared with depth.The relative error of the distance is controlled within 1.6%,which lays the foundation for the in-depth study of the tomato picking robot vision system.