Study On In-situ Crop Leaf Detection And Segmentation Via Deep Learning

China issued the “14th Five-Year Plan” for the Development of Agricultural and Rural Informatization in 2022,which included smart agriculture as a major direction for national agricultural and rural informatization development.Smart agriculture used big data and other methods to effectively integrate and penetrate China’s agricultural industry chain with other industries,helping to promote the development of China’s agricultural production towards efficiency,refinement,and sustainability,and further promoting the transformation and upgrading of the country’s agricultural industry.In-situ detection and instance segmentation of crop leaves are crucial measures for the development of smart agriculture.Real-time monitoring of crops provides timely information on the growth status and health of crops in the field,thus providing a basis for scientific management of farmland.However,crops grown naturally in the field suffer from severe occlusion and overlapping usually,and are easily affected by complex weather conditions.Traditional crop detection and instance segmentation methods mostly extract key information from leaf feature maps through image processing,which is time-consuming and labor-intensive.Deep learning-based methods,on the other hand,have become an efficient technical means for real-time monitoring of crop yields due to their advantages of high efficiency and high accuracy.Therefore,in order to improve the detection and segmentation performance of crops in complex conditions,we proposed a deep learningbased method for in-situ detection and segmentation of crop leaves.The main research contents in the paper are as follows:(1)In the process of in-situ detection of crop leaves,we addressed two problems.First,to address the problem of imperfect feature extraction for leaf targets,we added the Convolutional Block Attention Module(CBAM),which fused spatial and channel information,to the feature extraction network of the model.It enhanced the model’s feature extraction capabilities while also increasing the number of effective features.Second,to address the problem of accurately detecting overlapping or occluded target leaves,we designed the DIo U-NMS algorithm(Distance Intersection over Union-Non-Maximum Suppression)to replace the original NMS algorithm(Non-Maximum Suppression)in the post-processing of the model.It took the overlap,distance,and ratio between targets into considered,minimized the problems of missed and false detections of crop leaves in dense scenes to the greatest extent possible.(2)In the instance segmentation process of crop leaves,to address the problem that the Mask generated by the existing example segmentation model were difficult to accurately cover the dense target blades,we redesigned and optimized the structure of Mask,so that we obtained a more applicable Mask structure for the model.Few experimental results showed that the optimized instance segmentation architecture significantly avoided the problem of partial target missing caused by segmentation.The extensive experimental results of this paper showed that in the deep learning-based in-situ detection of crops,the proposed method achieved an average precision value of 95.7%for the mean Average Precision(m AP)at a threshold of 0.50(Io U=0.50),which was 2.9%higher than that of the official Faster R-CNN(Faster Region-based Convolutional Network)model and 7.0% higher than that of the YOLOv5(You Only Look Once v5),demonstrating better detection and classification performance.In the deep learning-based segmentation task,the optimized model in this paper achieved an m AP@0.50 value of 98.8%,which was1.8% higher than the official Mask RCNN(Mask Region-based Convolutional Network),effectively improving the actual segmentation effect of crop leaves in complex environments.In summary,the proposed method was reasonable and efficient for in-situ detection and segmentation of crop leaves,providing technical support for monitoring farmland in natural environments.