Research On Fast Dynamic Recognition And Location Of Overlapping Fruit For Apple Harvesting Robot

In China, fruit and vegetable harvesting still mostly rely on manual work, however, as rural population declining, aging population growing and some fruits and vegetables must be harvested within a short time, agricultural labor force is increasingly inadequate. Therefore,researching on fruit and vegetable harvesting robot and achieving automation of fruit and vegetable harvesting are imminent. Vision system as a core part of harvesting robot has a critical influence on robot harvesting.This paper is funded by the program of National Natural Science Foundation ‘Research on efficient machine-based harvesting method of apples based on fast visual servo control in multi illumination environment(31571571)’ and Doctoral Fund of the Ministry of Education ’Research on adaptive and accurate identification of common growing morphological fruit of apple picking robot(20133227110024)’. It is focused on fast dynamic recognition and positioning of overlapping apples in the orchards and it mainly covers the following:1、Image denoising and color characteristic analysis of overlapping apples. Due to a variety of factors, images captured from the orchard usually contain some noise, which is likely to cause interference to the subsequent image recognition. Thus, image denoising is the first step to identify the target fruit. In this paper, color image neighborhood average and median filter are used to remove noise in the image, then color characteristics of the low-noise image after denoising are analyzed, which lays the foundation of image segmentation.2、Overlapping apple image segmentation and improving. Segmentation is the basis of recognition, this paper compares three kinds of image segmentation method based on threshold and finally chooses improved R-G color segmentation algorithm to segment overlapping apple image after denoising. The algorithm increases the difference between red component and green component of image by conducting gamma conversion on red component of RGB image, which can ease over-segmentation or under-segmentation situation when red and green color difference between apple fruit and the background is not obvious. The combination method of mathematical morphology, holes filling and threshold area reservation is used to perfect the image after denoising.3、Fast dynamic recognition of overlapping apple image. Robot picking operations are usually carried out in the state of motion, so, dynamic method is tried in the article to identify apples, which obtains the dynamic characteristics of apple fruit by analyzing image sequences.After segmentation, maximum of the minimum distances between the points inside the target fruit and contour edge is calculated by fast generating distance function method, thecorresponding point of maximum is the center of the fruit. The radius is determined by the minimum value of the maximum distances between the center and contour edges in different directions. The template of matching is extracted by combining the center and radius. Then fitting the robot movement path according to each center of the image, estimating subsequent motion, narrowing the scope of subsequent image processing, and accelerating recognition speed.Finally, fast normalized cross correlation matching algorithm is used to recognize overlapping apples.4、Binocular camera calibration and depth information acquisition. Zhang Zhengyou plane calibration method is used to calibrate the camera and obtain its internal and external reference,the reprojection error of left and right camera is [0.22115,0.21346] and [0.24535,0.27693]respectively, and can meet the basic requirements of picking. Then compare and analyze region-based and feature-based stereo matching algorithm. Eventually, taking the centers of left and right apple image as matching feature points to conduct feature-based stereo matching and depth information is got according to the principle of triangulation.5、Experimental analysis of dynamic identification and location algorithm of overlapping apples. It can be seen from the experimental results that for a size of 320x240 pixels image with apple area accounts for about 18.5%, matching time is 0.181 s without optimization, after optimization, matching time is 0.094 s, which is improved by 48.1%. And the smaller of apple fruit area rate is, the more obvious effects of acceleration optimization will have, the more fruit overlapping portion is, the shorter time of matching recognition will need. Optimum distance of image acquisition is 350 ~ 600 mm, depth distance error is within 15 mm at this time, which can meet the requirements of harvesting in the general situation.