Research On Fruit Recognition And Location Based On Deep Learning Binocular Vision Technology

Emphasizing agriculture and solidifying the foundation is the foundation of Anmin.The issue of "three rural issues" has always been one of the key concerns of the state.While continuing to vigorously develop the industrial economy,constantly optimize the agricultural industrial structure,adjust industrial policies,and promote the progress and development of the agricultural industry.With the development and progress of modern science and technology,it is gradually moving towards the modernization,intelligence and automation of the agricultural industry.Fruit identification and positioning is the key to intelligent picking.Aiming at the problems that the background color of the green pepper and the plant are similar during the detection of green pepper fruit and the complex environmental background in the greenhouse,this paper takes the green pepper fruit in the greenhouse environment as the research goal and studies the target detection and positioning technology of the picking robot.The main research of this article is as follows:Firstly,the green pepper fruits in the greenhouse environment were studied,and the Mask-RCNN target detection algorithm was proposed to segment and identify the green pepper fruits.Study the Mask-RCNN network structure,and optimize the deep residual network and feature pyramid network structure on this basis.Secondly,using the green pepper images collected in the greenhouse environment,first use Labelme to label the original image data set to generate a mask image to calculate the reverse loss during the training process and optimize the parameters.And by comparing the annotated mask image with the prediction result of the mask,the performance of the training model used for instance segmentation is evaluated.Use ResNet-50 as the backbone network of the green pepper target detection model,set the IOU to 0.55,and use the transfer learning method to train the labeled data set.perform statistics on the loss values,and generate a training loss curve.The loss values can all converge and stabilize in one.Within range.Using 150 test set images for model performance verification,the results can be obtained through statistical analysis:the average accuracy rate(AP)is 0.944,the average average accuracy(mAP)is 0.927,and the average detection time is 1.2s/piece,which can achieve the fruit picking robot Requirements for real-time target detection.Then,study binocular camera calibration and three-dimensional matching algorithm.According to the camera imaging model,binocular camera imaging principle and depth calculation principle,it is determined to use Zhang Zhengyou calibration method to obtain the internal and external parameters of the binocular camera.The centroid-based feature point matching method is used to select appropriate matching constraints and similarity measurement functions for three-dimensional matching.Finally,use the ZED Stereo Camera binocular stereo camera,NVIDIA JETSON NANO embedded development board,display,tripod,etc.to form a binocular stereo vision system,and calibrate and stereo correct the system,and then obtain the fruit centroid coordinates and perform stereo matching,Finally,a space ranging experiment is carried out.The experiment shows that the error range between the actual measurement distance and the actual distance of the binocular camera is within 0～20mm,and the error range can basically meet the accuracy requirements of green pepper fruit picking.