Research On Green Fruit Recognition Based On Machine Vision In Near Color Background

Applying machine vision technology to early estimation of fruit yield and object recognition of fruit and vegetable picking robot can greatly reduce the manual operation in the process of planting and management,labor intensity and save labor costs.Also,it helps to realize the automation and intelligence of orchard management.The object of early estimation of fruits is immature green fruit,and the picking robot also needs to realize the automatic harvesting of green fruits such as fragrant pears and green apples.During the whole picking operation of the picking robot,the main problem and difficulty to be solved is the automatic detection and positioning of the fruit.However,the fruit itself is green,which is similar to the color of leaves and weeds.In addition,the fruit images collected in the natural environment suffer from uneven illumination,complex background,obscured branches and leaves,overlapping fruits,etc.Therefore,accurate identification of the green fruits in the near-color background is a key problem to be solved urgently.In this paper,immature peach,tomato,persimmon and citrus are taken as research objects.The detection methods of green fruits in natural scene are traditional feature extraction method and deep learning method based on convolution neural network.Based on the deep learning method,a fruit recognition vision system is designed,which lays a theoretical foundation and technical support for the subsequent development of early production estimation and picking robot for fruits and vegetables.The main research works of this paper are as follows:(1)Establishment of image data set of immature green fruits.Due to lack of publicly relevant data sets,images of four kinds of fruits were collected on the spot,including scenes such as forward light,back light,side light,occlusion,overlap,etc.In order to further enhance the diversity of data and avoid overfitting,the methods of horizontal mirror flip,CLAHE(contrast limited adaptive histogram equalization),and PCA(principal component analysis)Jittering were used to amplify the data.(2)Green peach recognition based on improved DRFI(discriminative regional feature integration)algorithm.The surface of mature fruits is often red,yellow,or other colors that are significantly different from the background.In contrast,the color of immature green peaches is highly similar to that of the background,which cannot be distinguished by color characteristics alone.This paper proposes to replace some of the features originally used in the DRFI saliency detection algorithm with the unique color,texture,and shape features of green peaches.Meanwhile,the corresponding parameters are adjusted to make it more suitable for calculating the saliency map of immature green peaches.Then,a fixed threshold is used to perform binary segmentation on the obtained DRFI saliency map,which reduces the false segmentation of the background area in the saliency map.Aiming at the situation that the fruits still stick to each other after the threshold segmentation,the watershed segmentation algorithm combining the control marker and distance transformation is used to separate them.The experimental results show that the accurate recognition rate of the method in the test set is 83.2%,the false detection rate is 8.7%,and the missed detection rate is 16.8%.The recognition results revealed that the proposed method in this paper can effectively solve the problems of color similarity,occlusion of branches and leaves,and overlapping of fruits.(3)Green fruit recognition based on YOLOv3(you only look once version3)algorithm.The YOLOv3 algorithm uses deep convolutional neural networks to extract the target features,and then directly classifies the targets,and predicts the location of the target in the image.It has high accuracy and fast detection speed.In this paper,the coordinate loss of YOLOv3 is modified as the GIoU(generalized intersection over union)loss to optimize the positioning results of the target.During training,K-means clustering analysis is carried out to generate anchor boxes of appropriate size for the data set.YOLOv3 network after modifying the loss function is applied to the recognition of four kinds of immature green fruits,namely,peach,tomato,persimmon and citrus,by using the method of transfer learning.Finally,the performance of the algorithm is verified.The experimental result shows that: under the same conditions,the accurate recognition rate of YOLOv3-L is 98.0%,the false detection rate is 1.0%,and the missed detection rate is 2.0%.The recognition effect is better than the improved DRFI combined with the watershed method.The mAP(mean average precision)of the peach image set detected by YOLOv3-L network is 92.71%,the mAP of tomato image set is 93.98%,the mAP of persimmon image set is 84.51%,and the mAP of citrus image set is 79.00%.It shows that YOLOv3 network has high detection accuracy and strong universality.Experiments on peach image sets in different orchards,and tomato image sets in greenhouses and natural environments show that the YOLOv3 target detection network has good generalization ability.In general,the larger the sample size and the better the diversity of data set,the stronger the generalization ability of the trained model.(4)Design and implementation of fruit recognition system.In order to make it more convenient for orchard growers and managers,a fruit recognition system has been developed through PyQt5,TensorFlow and OpenCV.The human-computer interaction system includes image loading,model selection,result displaying,fruit number statistics and other functions,so that the operator can clearly and intuitively understand the situation of fruit recognition by simply clicking a few buttons.