Study On Detection Of Pinus Armandii Cones In Natural Environment Based On Machine Vision

The application of machine vision technology to automated harvesting equipment can help solve the problems of high labor costs and shortage of labor force.However,fruit background in natural environment is complex,and machine harvesting is affected by many factors,such as occlusion,light intensity and number of targets.Therefore,the harvesting robot requires high precision of fruit detection and harvesting.In this paper,the cone of Pinus armandii in Shaanxi Province was used as the experimental object,and the detection of green pinecones during harvesting based on machine vision was studied.The main research work of this paper is as follows:(1)For the detection difficulty of similar color of pinecones and background,this paper proposes a pinecone object detection algorithm based on improved saliency detection via graph-based manifold ranking.Firstly,the image preprocessing uses the bilateral filtering algorithm to reduce the detail interferences such as pine leaves,and the Laplacian algorithm to enhance the characteristics of pinecone region in the image.Secondly,the preliminary features of pinecone regions in the image are obtained by combining SLIC super-pixel segmentation algorithm and saliency detection via graphbased manifold ranking,and the binary image of pinecone regions is obtained by OTSU algorithm and morphological processing of close operation and open operation.Subsequently,the multiple pinecone regions are segmented by the selected concave points in the contour.Finally,the corresponding regions are filtered and pinecones in the original image are marked by the coordinates of the rectangular boxes surrounding the regions.The experimental results show that the traditional algorithm proposed in this paper has some limitations in application,which is greatly affected by the factors such as light,fruit number,occlusion and distance,etc.However,in an ideal environment,the precision of the algorithm for pinecone detection reaches 82.26%,and the recall reaches 92.49%.The algorithm can recognize pinecones when pinecones are similar in color to the background,which provides a technical basis for the traditional algorithm to detect green pinecones.(2)For the pinecone detection difficulties in natural environment,such as occlusion,large number and small targets,this paper proposes a pinecone detection algorithm based on M-RFBNet model,which is an improvement of RFBNet-E model.It adds feature fusion to the shallow network,enriches the feature information of small targets,and enhances the detection effect of the model on small targets.CBAM attention module is added at the end of each RFB module output to optimize the problem of memory overflow in model training,and optimize weights of feature channels.With adding residual structures between deep feature layers,the feature information of large targets is further supplemented to enhance the detection effect of large targets.The experimental results show that the mAP of M-RFBNet model detection on the pinecone dataset in this paper reaches 97.56%,and the detection speed reaches 18fps.Compared with YOLOv5s,SSD,Faster R-CNN and other models,it has the best comprehensive performance.It has good robustness in complex harvesting scenarios,and can meet the application needs of machine picking and can complete the prediction of pinecone yield,etc.