Recognition And Segmentation Of Green Apple Targets In Natural Environments

Effective detection of fruit targets with the similar-background skin color in natural environment has great significance for fruit quality improvement,orchard management optimization and reasonable market resource allocation.In order to solve the problems caused by similar colors between fruit skins and backgrounds,varying illumination,and branch and leaf occlusion,the green apple targets planted in Yangling,Shanxi were taken as the research objects in this study.And a few recognition,segmentation and reconstruction methods of green apple fruits under random illumination conditions and different growth statuses were proposed to meet the needs of orchard workers for fruit growth status monitoring and intelligent estimation.The major research contents and results of this study are as follows:(1)Image highlights and shadows caused by random illumination can make green apple target difficult to identify.In response to this problem,the five image enhancement methods including the fuzzy set theory,the SSR algorithm(Single-scale Retinex),the MSR algorithm(Multi-scale Retinex),the MSRCR algorithm(Multi-scale Retinex with color restoration)and the illumination invariance theory were compared to select the appropriate method for enhancement preprocessing of green apple images.Experimental results showed that,under visible light conditions,the average mean square error of the fuzzy set theory,the MSRCR algorithm and the illumination invariance theory were 17169.32,3428.00 and 3161.64,respectively.The average peak signal-to-noise ratio and average structural similarity of the three image enhancement methods were 6.11 dB,13.48 dB and 13.09 dB,and 0.00,0.79 and 0.66,respectively,which were superior to those of the SSR algorithm and the MSR algorithm.In short,the three image enhancement methods could effectively improve the influence of image highlights and shadows on fruit target recognition.(2)Aiming to solve the difficulty of fruit target recognition resulted from many disturbance factors such as green skin color,light randomness and branch and leaf occlusion,the four recognition methods of green apples by fusing the fuzzy set theory and the MR algorithm(Manifold ranking),employing the AIM algorithm(Attention-based on information maximization)and the illumination invariance theory,combining the MSRCR algorithm and the Mean shift algorithm,and utilizing the improved GrabCut model were proposed.In this study,100,100,500 and 200 green apple images were used as test sets for four fruit recognition methods,and 50 green apple images were applied for verification.Experimental results showed that,in the test sets,the average recognition accuracies of the four fruit recognition methods were 90.87%,86.99%,86.67% and 89.50%,respectively.The average false positive rate and false negative errors were 0.53%,0.97%,0.58% and 1.42%,and 7.46%,10.20%,11.64% and 5.29%,respectively.Besides,in the verification set,the average recognition accuracies of the four recognition methods were 83.38%,84.06%,80.13% and 89.06%,respectively.The average false positive errors were 2.69%,1.22%,0.29% and 2.82%.And the average false negative errors were 10.37%,13.27%.20.52% and 3.33%.Compared with the traditional visual attention mechanism models and the clustering algorithms,the above methods could significantly improve the recognition accuracies and obtain better recognition effects of green apples.(3)In general,the multi-overlapping apple targets in natural environment are often misjudged as single fruit target in the recognition process.To solve this misjudgment problem,a segmentation method of green and multi-overlapping apple targets relying on the Ncut algorithm was studied.Based on the improved GrabCut model,the Ncut algorithm was introduced to segment the multi-overlapping fruit targets in green apple images.Experimental results showed that 93.42% of the green apples could be effectively segmented.The method could segment the green and multi-overlapping apple targets into separate fruit targets for subsequent accurate reconstruction.(4)Due to the misjudgment of growth position and fruit shape information of green apples arising from branch,leaf and fruit occlusion and mixed occlusion,the fruit reconstruction method based on the three-point circle fitting method was proposed.For the separating green apples after segmentaion,the three-point circle fitting method was developed to achieve effective recovery of the occluded fruit areas.Experimental results showed that the average reconstruction error was 7.37%,which indicated that the algorithm could effectively reconstruct the green apples in natural environment,and achieve the accurate localization of green apples under the influence of branch,leaf and mutual fruit occlusion.(5)The recognition and segmentation software system of the green apple targets in natural environment was designed relaying on the GUI compilation toolbox(Graphical user interface)in Matlab.The software mainly includes a recognition module,a segmentation and reconstruction module,and four recognition process viewing modules.After testing,the software could realize the comparison of the four recognition methods,the segmentation of green and multi-overlapping apple targets,and the reconstruction of each fruit target.This system had the advantages of simple operation,and could display the detailed processes and results intuitively.