Research On Blade Image Recognition Based On Deep Learning And Feature Fusion

Leaf image pattern analysis is an important application research problem in computer vision,which plays an important role in plant classification,crop phenotype,environmental ecology and other research fields.Due to the huge amount of plant species and the endless emerge of new species,let alone the high similarity of leaf visual patterns of many species and the natural deformation of plant leaves of the same species,thereby the problem of leaf image pattern recognition is still a challenge.The recognition of fine-grained cultivars of the same crop is a more challenging pattern recognition problem,which requires high-performance recognition algorithms to meet the practical application requirements of precision agriculture.In order to solve the problem of coarse-grained species recognition and fine-grained cultivars recognition of leaf images,this paper focuses on leaf image analysis based on deep learning and feature fusion.Hand-crafted features of images can describe the underlying visual information of images,rather than depending on training data,and be interpretable.Although deep features depend on training data and have poor interpretability,they can effectively obtain abstract semantic feature information of images.In view of the strong complementarity of hand-crafted features and deep features,the research work of fusing these two types of features has attracted the attention of researchers in recent years.However,most of the proposed fusion methods have the shortcomings of too many model parameters,not end-to-end,and poor generalization ability.This paper proposes a deep network: HDFF,which integrates manual feature information by means of knowledge distillation to solve the coarse-grained recognition problem of leaf images.The network not only gives full play to the complementarity of hand-crafted features to deep features,but also only retains an independent deep network structure when the model is deployed,which can complete the end-to-end prediction task without carrying and executing manual feature extractors,thus saving parameters and improving the prediction speed.The HDFF model consists of two parts: a deep feature classification network and a manual feature classification network.In the forward process of training the deep network and class label information,the features of the deep network are copied and dynamically embedded into the manual feature network(the process is equivalent to knowledge flowing from the deep network to the hand-crafted network),and the fused features are used to train the classifier.Then,the prediction probability vector is used to form a dynamic mutual learning process with the prediction probability of the deep network in the way of knowledge distillation,which is equivalent to the bidirectional flow of knowledge in the two networks.In this paper,Res Net50 and Dense Net121 are selected as deep backbone networks,and the features of leaf shape method Ho GCV and leaf texture method PRICo LBP are used as hand-crafted feature methods.The HDFF model achieves 96.3% and 83.2% accuracy on Leaf220 and MEW2012 datasets respectively,which are 2.4% and 2.0% higher than the baseline model without fusion.It is proved that the hand-crafted features and deep features are complementary,and the HDFF fusion method can improve the accuracy of leaf recognition.It also expands a new idea for the future research work of the fusion of hand-crafted features and deep features.For fine-grained recognition tasks,it is difficult to obtain satisfactory recognition results by using the leaf of a single part of the plant.This paper developed the existing method of fusing the leaf features of the upper,middle and lower parts of the plant to improve the accuracy of variety recognition,and proposed a deep feature fusion network of triplet images based on contrastive learning: CLFF.The fine-grained variety recognition of plant leaves is extremely challenging due to the great difficulty of classification,the small difference between classes,and the problem of small training samples caused by the cost of data labeling,which make the task extremely challenging.In this paper,the deep learning feature joint training of the triplet leaf image composed of the upper,middle and lower leaves of the soybean plant is proposed.The common information feature of the triplet image is calculated,and then orthogonal fusion is performed with the differentiated features of each subgraph.The CLFF model not only achieves an accuracy of 92.6% on Soy Cultivar200,a soybean variety identification database,which is much higher than that of the existing methods,but also alleviates the overfitting problem that may be caused by too few datasets,which again verifies the effectiveness of multi-part leaf complementary information for fine-grained variety identification.Compared with previous multi-part feature fusion methods,the CLFF model first achieves better classification performance.Secondly,the model has end-toend advantages.Last but not least,it has better robustness and generalization ability.