Study On Deep Learning Based Apple Detection And Segmentation Methods

For developing countries,intelligent agriculture is the main component of an intelligent economy and the main way for developing countries to eliminate poverty,realize the advantage of late development,catch up in economic development and realize the catch-up strategy.In recent years,with the rapid development of China’s Internet of things,artificial intelligence,big data,intelligent equipment,and other fields,and the leapfrog progress of various key technologies applied in the field of agricultural planting,China’s agricultural industry has been transformed into intelligent,digital,efficient and accurate.In the process of Apple planting and management,the most time-consuming and laborious work is fruit harvesting.Before fruit harvesting,accurate and effective detection and positioning of apples are very important for fruit automated harvesting.With the increase in labor costs and the reduction of the number of skilled workers,the cost of apple picking becomes more and more expensive.To solve the problem of apple orchard design and automatic fruit picking,many robot algorithms are widely used in orchard design and automatic fruit picking.However,due to the complex background of actual apple orchards,factors such as light fluctuation,dense distribution of fruits,fruits blocked by branches and leaves,fruits overlapping,etc.,all have a certain influence on target detection,which brings difficulties and challenges to accurate identification of fruits.With the booming development of deep learning,when facing real scenes,deep learning often has a more robust performance.Therefore,how to apply deep learning technology to the automatic picking task of orchards,and at the same time further improve algorithm performance and automatic picking efficiency,has become the focus of smart agriculture-related researchers.In this thesis,the apple is taken as the research object,and deep learning technology is used to realize the automatic detection and positioning of apples in the natural environment,which has important practical significance for intelligent management and automatic picking of an apple orchard in the natural environment.The main work of this thesis has two aspects as follows.In the first work,this thesis proposes an apple detection and segmentation model based on aspect ratio constraint and channel attention mechanism named SE-Mask R-CNN.firstly,the detector network should pay more attention to the feature maps containing more information about apples,so this method introduces the squeeze-and-excitation module in the ResNet-50 backbone that The available computational resources are allocated to the most informative feature maps in the form of channels.Secondly,this method optimizes the regression loss of bounding box with aspect ratio to make Mask R-CNN more suitable for apple detection and segmentation,which can assist the regression of bounding box by deforming the shape of bounding box to apple in the training phase.Finally,to improve the detection performance in complex backgrounds such as overlap and occlusion,the method replaces the traditional NMS in Mask R-CNN with Soft-NMS,which can remove the redundant bounding boxes and reasonably obtain the correct detection results.In the second work,the method proposes a lightweight apple detection model based on a mask-guided attention mechanism.First,the method in this thesis uses a lightweight backbone network Efficientnet-B0,which can surpass the accuracy of state-of-the-art methods with fewer parameters and operations,and greatly reduces the number of parameters and the complexity of operations in the feature extraction network.Second,the method establishes a single-stage object detection network with an adaptive training sample selection strategy(ATSS)for apple detection from feature maps,which proves that the performance of the target detection model is largely influenced by the positive and negative sample selection strategies,and the adaptive training sample selection strategy can adaptively select positive and negative samples based on the statistical characteristics of the object,which enables the model to surpass the performance of another single The model outperforms other single-stage object detection networks.Finally,a new branch named the Mask-Guided Attention(MGA)branch is designed in this thesis,which introduces the mask information of apples and enhances the eigenvalues of the pixels where the apples are located.It is experimentally demonstrated that the mask-guided attention mechanism proposed in this thesis can guide the learning of other branches and improve the differentiation between apples and backgrounds.The experiments conducted in this thesis are on the Minneapple dataset,and the excellent performance of the work in this thesis is verified by ablation experiments and comparison tests.