Research On Mulberry Leaf Picking Location Based On Deep Learning

China’s sericulture industry has a long history and enjoys a high reputation worldwide.In recent years,Guangxi has benefited from its unique geographical and climatic advantages and the support of national policies,vigorously developing the mulberry and silkworm industry,and has consistently ranked first in the country in silkworm production for many years.The mulberry and silkworm industry has made significant contributions to regional economic development,farmers’ income increase,poverty alleviation and rural revitalization.Although some mechanized and semi-automatic mulberry leaf picking equipment has been developed for the mulberry leaf picking process,it is still not completely separated from manual labor,and there are strong seasonal picking and high labor costs,which have become an important factor limiting the development of the mulberry industry.During the feeding process of young silkworms,it is necessary to pick young and suitable mulberry leaves one by one,which is complex and labor-intensive.As traditional agriculture moves towards emerging smart agriculture,replacing traditional manual and mechanized harvesting with intelligent harvesting has become a future development trend.To achieve intelligent picking of mulberry leaves and enable machines to automatically identify picking points on mulberry branches without manual intervention,the key step is to obtain the location information of the picking points of mulberry leaves.This article takes mulberry trees in natural environments as the research object,and combines Mask RCNN instance segmentation algorithm and skeletonization processing methods to study the recognition and localization of picking points for mulberry leaves.The main research content is as follows:(1)Improve Mask RCNN’s mulberry leaf node region recognition model.Building a node area model for mulberry leaves is the "first step" in achieving the positioning of mulberry leaf picking points.Firstly,Mask RCNN was selected as the basic network to further improve the accuracy of the algorithm in identifying and segmenting mulberry leaf node regions.The backbone network,RPN network,and anchor box ratio were improved.The backbone network Res Net is replaced by Res Ne Xt network,and a bottom-up enhancement path is introduced into the feature pyramid network,so that the edge,shape and other information contained in the bottom of the pyramid are further integrated with the high-level information of the pyramid.Design a multi-scale regional recommendation network,design convolutional kernels of different sizes for each feature layer after path enhancement,and overlay them.Improve the aspect ratio of the anchor box to match the actual aspect ratio of the target.The experimental results show that the improved network model has m AP and recall rates of 89.1% and 67.1%for detecting mulberry leaf node regions,respectively,which are 2.8% and 3.9% higher than the original Mask RCNN,effectively enhancing the network’s detection performance and reducing missed detections.(2)A picking point localization method based on traditional image processing.Using traditional image processing techniques to locate the corresponding points in the mulberry leaf node area as picking points is the "second step" in achieving mulberry leaf picking point positioning.Firstly,statistical analysis is conducted on a large number of segmented mulberry leaf node regions,which can be divided into two categories based on their shapes: Y-shaped "Y-shaped" node regions and rectangular "rectangular" node regions.Then,using the refinement algorithm to skeleton these two types of node regions,a picking point localization method based on the vector angle size is designed for the bones in the "Y-shaped" node region;Directly extract the centroid of the bones in the "rectangular" node area as the picking point.Experiments have shown that both types of node regions have high accuracy in locating picking points,meeting practical needs.(3)The fusion method of dual perspective picking points.A method based on dual view picking point fusion is proposed to address the problem of occlusion in the node area and picking points of mulberry leaves identified from a single perspective.Firstly,two cameras are used to collect images of mulberry trees on the front and with a deviation of 25 °.The proposed picking point positioning method is used to locate the mulberry leaf picking points in the two images.Then,using the single pixel skeleton of the mulberry branch as a bridge,the picking point coordinates that are occluded in the left camera image but can be correctly identified and located in the right camera image are mapped back to the original left camera image,achieving the goal of fully integrating all picking points in the two perspective images into one image,Ensure that picking points are not missed.This article uses an improved model to locate the picking points of mulberry leaves,providing visual guidance for intelligent picking of mulberry leaves.At the same time,a dual perspective fusion method for picking points is proposed,which also provides new ideas for solving the occlusion problem in the intelligent picking process of other fruits or leaves.