Recognition And Localization Method Of Apple Target Under Conditions Of Overlapping And Occlusion

For the vision system of apple picking robots, recognition and localization of apple target under conditions of overlapping and occlusion are key problems which must be solved.Whether fruit target and picking point can be identified accurately and quickly or not will affect efficiency of picking robot directly. In order to achieve the goal of fast recognition and accurate localization of apple targets under the conditions mentioned above, this study proposed different algorithms. The main research contents and conclusions are as follows:(1) In order to select the best segmentation method to segment apple image, the performance of four methods including fusion of muti-color space algorithm, chromatic aberration algorithm, fuzzy C means algorithm and K-means algorithm were compared.Conclusion could be drawn that the average segmentation error and average overlap ratio of K-means clustering algorithm was 2.245% and 97.660%, respectively. The average run time was 0.818 s. The performance of K-means algorithm is the best among these four algorithms,which could extract apple target from image stably, accurately and efficiently. Hence K-means clustering algorithm was selected to segment image in this study.(2) Segmentation and reconstruction of 2 overlapped apple method which based on Normalized Cut(Ncut) algorithm and Spline interpolation algorithm were proposed.Overlapping apple targets affects the precision of targets localization and the execution of picking tasks seriously. In order to segment and reconstruct 2 overlapped apple targets, firstly,after K-means segmentation and image pre-processing, Ncut spectral clustering algorithm was used to segment the image. Then contours of each apple were extracted. After that,Spline interpolation method was used to reconstruct the contour of occluded apple. The experimental result showed that Ncut algorithm could keep the complete contour information of unblocked apples and thus improving segmentation accuracy. For unblocked apple targets,the average segmentation error and average overlap ratio was 3.147% and 96.081%,respectively. For occluded apple targets, the average segmentation error and average overlap ratio was 5.243% and 93.709%, respectively, which indicated that the method could segmentand reconstruct 2 overlapped apples accurately.(3) A method based on a circle determined by three points algorithm was proposed to locate occluded apple targets. Under natural scene, apple targets often blocked by leaves and branches, which lead that apple picking robot cannot locate apple targets quickly and accurately and affect implementing picking tasks seriously. To solve this problem, firstly,contour and convex hull of apple was combined to extract real apple edge. Then three points on real edge were select to determine a circle, which was used to locate occluded apples. In order to avoid problem of different candidate points causing quite different parameter estimation, select points 100 times and 100 circles could be got. Removing too big and too small circles, the average circle center coordinates and radius of rest circles was used to locate the occluded apple target. The experimental result showed that the average segmentation error, average false positive rate, average false negative rate and average overlap index was 8.259%, 1.235%, 8.227% and 88.921%, respectively, which indicated that this algorithm could locate occluded apple targets accurately.(4) Method of localization picking points of single and unblocked apple targets based on contour smoothing algorithm and moment of inertia algorithm was proposed. Localization of picking points of single and unblocked apple target is the basis of that of apples under overlapping and occlusion. In this study, convex hull of apple was used to replace apple contour, which could smooth contour and improve accuracy and efficiency. Moment of inertia algorithm was use to extracted symmetry axes of apple and locate picking points of apples. The experimental result showed that the average location error and average run time of the algorithm was 5.617°and 3.743 ms, respectively, which indicated that the algorithm could locate picking points of single and unblocked apples accurately and efficiently.(5) GUI compiler toolbox in MATLAB was used to design software system of recognition and localization of apple target under conditions of overlapping and occlusion.The software includes four modules, that is, segmentation algorithm comparison module, 2overlapped apple targets segmentation and reconstruction module, recognition and localization of occluded apples module and localization picking points of single and unblocked apple targets module. This system is simple and intuitive, and it possesses beautiful interface.