Research On Crop Recognition Method Based On Deep Learning And Visual Saliency Under Weedy Condition

With the enhancement of people's awareness of environmental protection and the increase of attention on food safety,the government has formulated more and more strict environmental protection policies to limit the use of various chemical herbicides.Reducing the dependence on herbicides for weed control is one of the main challenges of sustainable agriculture.Robotic herbicides is a feasible method,which can not only reduce the environmental load of chemical weeding but also maintain a high operating efficiency.In the face of a complex field environment,one of the key technologies of robotic weeding is the automatic recognition and localization of crop plants and weeds.Based on the current technological progress and its shortcomings,this research investigates the following aspects to deal with the transplanted crop recognition problem under weedy conditions:First of all,this research proposes a visual-attention-based method of crop recognition and establishes a field images dataset collected under weedy conditions,for demonstrating the saliency of crop plants and fulfilling accurate crop/weed segmentation.The dataset contains 788 cauliflower images with high weed infestation levels which are precisely annotated.According to the visual attention mechanism of the human visual system,the method uses a convolutional neural network to segment crop plants from overlapping weeds.In this network,a Res Net-10 is used as the backbone,and side outputs and short connection are introduced for multi-scale feature fusion.The Adaptive Affinity Fields method(AAF)is adopted to improve the segmentation at object boundaries and for fine structures.The experimental results show that the method can accurately separate the crops from weeds and soil,and the mean absolute error(MAE)is less than 0.5%,and the F-measure score is more than 97%.In terms of efficiency,the method can process a frame of image within 5.9 ms when accelerated by a GPU.Secondly,this paper proposes a fast crop recognition method based on a binary fusion tree structure.A lightweight fully convolution neural network is designed based on the saliency of crop plants.The network adopts a binary fusion tree structure for efficient multi-scale and multilevel feature fusion in a top-down way.A bipolar distribution loss is proposed to further improve the boundary and fine structure segmentation results.The experimental results show that the performance of this method is excellent,with MAE less than 0.4%,the F-measure score exceeding about 98%,and the intersection over union score over 96%.The processing speed of this method on the embedded computer of Jetson TX2 is over 26 Hz,which can meet the requirements of real-time operation in the field.Finally,this paper proposes a unified model for crop recognition and stem localization based on cross-task feature fusion.The model has a shared backbone network and two subnets corresponding to crop recognition and stem localization.The backbone extracts the common and fundamental features,while the two subnets perform the crop recognition and stem detection tasks respectively.According to the difference in the tasks of the two subnets,a feature fusion structure with varied span is designed.Besides,a cross-task feature fusion strategy is used to improve the stem detection performance by introducing the top-down guidance from the segmentation subnet to the stem detection subnet.The experimental results show that the proposed model can achieve outstanding crop stem detection accuracy and good crop recognition performance.