Research On Identification And Positioning Method Of Flat Jujube Picking Based On Deep Learning

Flat jujube is a new variety of jujube obtained from the grafting of pear jujube and sour jujube.It has fresh and crisp taste,high sugar content,and has high nutritional value and economic value.With the increasing number of orchards,the automatic picking of fruits in the field has become the essential way for the development of agricultural intelligence.Due to the complexity of the natural environment and the growth characteristics of the fruit,the detection model of fruit in the field has problems such as poor accuracy,low efficiency,difficult deployment,and easy to produce omission and false detection.So,this study takes the field flat jujube as the research object,it uses the technology of target detection based on the deep learning and binocular stereo vision technology to identify and make three-dimensional spatial positioning,and deploys the improved model for lightweight on PC and mobile terminals,so as to provide technical support for the automatic picking of flat jujube in the field.The specific research work and the conclusions are as follows:(1)The data set of field flat jujube was constructed and balanced,the hyperparameters were selected based on YOLOv5 network combined with transfer learning.In view of the few flat jujube images in the field,and the target detection algorithm based on deep learning required massive high-quality data to play its advantages,so this dissertation built the data set of field flat jujube.Meanwhile,this dissertation made preprocessing and data balance by guiding the filter to fog and seven kinds of image amplification methods.Due to the limitations of the number of CPU cores and GPU memory,the maximum values of workers and batch size were set to 4 and 64.In order to screen the optimal combination of other hyperparameters,six sets of models with different learning rates and three optimizers(SGD,Adam,Adam W)were selected for pre-training,and the impact on model performance was discussed.The results of the study showed that,the transfer learning used the trained weight parameters can greatly reduce the training time of model and speed up the convergence rate,and the convergence can be realized by about 50 rounds,which takes0.328-0.336 hours.Based on the basic idea of the method of control variable,six sets of comparative experiments with different learning rates can be obtained: when the learning rate was 0.01,the model had the highest accuracy and the least loss,and the m AP on the test set reaches 95.4%.Through three optimizers with four different learning rates,when the SGD optimizer and 0.01 learning rate were selected,the model had the highest accuracy and the fastest convergence rate.Meanwhile,the m AP and time consumption were 95.7% and 0.329 hours,respectively,with the best model performance.(2)Research on high-precision flat jujube identification based on BotNet network and adaptive spatial feature fusion.On the basis of the analysis of the current fruit identification model and positioning method,it was concluded that the problems of identifying flat jujube in the field had four difficulties: near-color background,cluster growth,oblate body and small target identification,which leaded to the problems of low accuracy,difficult identification and easy to miss and false detection in the actual detection.This dissertation proposed a high-precision improved network based on Bo TNet network and adaptive spatial feature fusion,and used ablation experiments and mainstream one-stage target detection algorithms to verify the model performance.Also,this dissertation selected the field crisp persimmon data set with the same cluster growth and oblate fruit to verify the generalization performance of the model.In the part of backbone network,it borrowed from the Bo TNet network structure and introduced the mixed attention mechanism modules in different positions of the network to enhance the feature extraction ability.After obtaining the feature map of different scales,the adaptive spatial feature fusion(ASFF)module was introduced into the neck network to realize the feature fusion of different scales and further enhanced the detection performance of the model.The experimental results showed that the precision,recall,F1-score,m AP,and m AP.5:.95 of the high-precision improved network reached 94%,92.2%,93%,97.8% and 89.1%,respectively.Also,these data were 3.2%,3.6%,3%,2.1%,and 8.6% higher than the original YOLOv5 s network,respectively.To verify the generalization of the improved network,after the training of improved network on the field crisp persimmon data set,the m AP and F1-score reached 96.9% and 0.93,respectively,and the m AP and F1-score of single classification detection reached 99.4% and 0.98.The experimental results showed that the improved network had high recognition accuracy for both field flat jujube and crisp persimmon,and the model had good robustness and generalization.(3)Study on lightweight flat jujube identification based on GhostNet network and deep separable convolution.In view of the problems of low efficiency and difficult deployment of existing target detection algorithm,a lightweight improved network based on GhostNet network and bidirectional characteristic gold tower was proposed.The network screened the appropriate multi-scale detection structure by adjusting the number of target detection layers,it integrated bi-directional feature pyramid network(Bi FPN)structure to realize multi-scale feature fusion,and introduced the ghost module to realize the lightweight of the model.The experimental results showed that the m AP of the lightweight improved network was 97.7%,and the recognition rate of mature and immature jujube reached 98% and 96.4%,respectively.Compared with the YOLOv5 s benchmark network,the m AP increased by 2 percentage points on the basis of the number of parameters decreased by 49.15%,and the model size was compressed from 14.4MB to 7.1MB,which improved the model detection accuracy and reduced the complexity of the model.On this basis,the depth separable convolution was introduced,and the C3 module was improved to reduce parameter redundancy and improve the rate of classification identification.At the same time,the loss function and non-maximum suppression were optimized to determine the best detection box,and the post-processing process was optimized to further improve the real-time improvement of the network.The experimental results showed that the m AP of the improved network was 97.4% which decreased by 0.3percentage points,but the model size was further compressed to 5.4MB,the detection time was compressed from 4.2ms to 3.5ms,and the detection speed increased by 16.67 percentage points.It could get the results by comparative experiment with mainstream one-stage target detection algorithms: m AP increased by 1.7%(YOLOv5s),23.5%(YOLOv5n),9%(YOLOx-s),5.7%(YOLOv4)and 6.7%(YOLOv3),the model size reduced to 62.5%,84.3%,97.8% and 97.7%,except slightly less than 3.8MB of YOLOv5 n,respectively.The method of this dissertation had a good effect of identification on small targets and blocked targets in the background of near-color containing leaves and crowns.On the basis of maintaining an increased accuracy,it greatly reduced the number of model parameters and floating points,improved the real-time detection efficiency,and provided a reference for the later deployment to the mobile terminal.(4)Study on 3D spatial positioning method of field flat jujube based on binocular stereo vision technology.First of all,the camera calibration method was used to obtain the internal and external parameters,and the Bouguet algorithm and image fusion were used to reduce the error caused by the lens distortion.The disparity map was obtained by stereo matching based on the SGBM algorithm and pole line constraints,and the spatial position of the field flat jujube was calculated based on the re-projection matrix and the least square method.The experimental results showed that the proposed method had a small error of positioning in X and Y direction,the error in Z direction and the object distance was slightly larger,the average absolute error was within 16 mm,and the relative error of positioning was less than 3%,which basically met the requirements of positioning accuracy of flat jujube and crisp persimmon in the field.(5)This dissertation developed a simple and easy-to-use and functional application of PC terminal detection system and mobile terminal APP to realize the lightweight deployment of the model.In the PC terminal application development,based on the high-precision improvement network,the detection software system with field jujube picture detection,video detection and the detection camera was designed by using Py Charm development platform,Py Qt5 development software and Qt Designer visualization interface development tools.In the lightweight deployment of mobile terminal,based on the lightweight improved network and the Android Studio development tools,the quantification of fp16 and int8 were used in Tensor Flow Lite format,and the real-time application of field jujube detection for Android system was developed.The lightweight deployment of the model not only reduced the dependence of the model on the hardware,but also improved the practicability of the model,which was conducive to the real-time management of the orchard planting managers on the orchard and improved the management efficiency.