Apple Detection And Counting Method Based On YOLO Light Network And Trunk Tracking

Accurate fruit number is important for producers to make appropriate decisions in production management.However,the number of apple fruits in commercial orchards is mainly estimated by manual counting,which is time-consuming and labor-intensive.Although some machine vision-based fruit counting algorithms have been developed,these algorithms are implemented by directly tracking the fruit.Due to the small size of the fruit and the high similarity of individual appearance,the tracking error is large and even loss the target.Consider of the trunk volume is generally larger than the fruit,and it can appear clearly in the video.Therefore,this study proposed an automatic video processing method based on single target tracking of the trunk to achieve higher accuracy and faster speed than the current fruit counting method based on multi-target tracking of fruits.The main research contents and conclusions are as follows:(1)Dataset acquisition and construction based on Real Sense camera.According to the characteristics of modern planting mode and apple growth,the shooting methods and data collection time were determined.The characteristics of apple agronomic production were analyzed,and the image acquisition equipment based on Realsen D435 camera and the image acquisition equipment composed of Realsen D435 camera,laptop and remote control car were determined.Analyze the target object categories of fruit detection and trunk tracking,determine the marked objects of the dataset as "apple" and "trunk",and study the object labeling method and precautions based on Label Img;according to the needs of target detection,the data is randomly divided according to the proportion collected,discussed and identified data augmentation methods for detection datasets to improve model robustness.Finally,according to the requirements of the YOLO network,the marked file format is converted and a dataset is made to train the deep learning network.(2)Research on target detection method based on YOLO light network.In order to select a detection model suitable for apples and tree trunks in the natural environment in the field,the network structures of YOLOv3-tiny,YOLOv4-tiny and YOLOv5 s were compared and analyzed,and the model was trained based on the same data set and tested on the same test set.Finally,10 orchard videos were randomly selected to verify the effect of the selected network model.The results show that the m AP of the YOLOv4-tiny model is 96.4%,and it takes 16 ms to detect an image,outperforming the other two networks.The AP value of this network reaches 94.2% in apple detection,which is 7.8% higher than that of YOLOv3-tiny,and only 1.2% lower than that of YOLOv5 s network,and there is no serious fruit missed detection in YOLOv5 s network.The AP value of this network reaches 98.6% in trunk detection,which is 3.9% higher than YOLOv5 s and only 0.2% lower than YOLOv3-tiny.The average R value of the model in the video is 94.95%,which can better detect the target in the video.(3)Research on fruit tracking method based on single target tracking of tree trunk.Aiming at the movement characteristics of video target in orchard tree row,this paper proposes an idea of fruit tracking based on tree trunk tracking.Compared with the results of tree trunk tracking based on target tracking network,a tree trunk tracking method based on Euclidean distance matching is proposed to calculate video motion displacement.Based on video motion displacement,a bidirectional matching strategy based on Euclidean distance is proposed to solve the problem of fruit ID transformation caused by fruit overlapping or close distance.A strategy for abnormal judgment of correct matching relationship is proposed to reduce the error probability of fruit matching.A scheme of secondary matching is proposed for the fruits that fail to match,and the video motion displacement is updated.Based on the Io U matching principle,secondary matching is performed on the fruits that fail to match,and the corresponding relationship between the fruits is increased.Finally,the developed fruit tracking method was used for experimentation in video,and finally obtained 3.7% IDSR value,90.9% and 94.4% MOTA and MOTP values.(4)Research on fruit counting method based on detection model and tracking algorithm.Research the fusion method of target detection and fruit tracking,and design a fruit counting pipeline.According to the characteristics of tree line video motion,the method to improve the processing speed of the algorithm is studied.Aiming at the problem of over-recognition of some non-counting target fruits caused by deep learning method,the reasons for the misrecognition of some FP fruits are discussed,and the removal method is explored to remove the detected fruits that do not need to be counted as much as possible.In order to improve the counting accuracy,the video frame fruit categories were divided,and the fruit classification judgment criteria were determined.Different counting strategies are formulated for fruits under each category,and appropriate fruit counting methods are formulated for different video frames.The video fruit counting algorithm was developed by integrating the target detection algorithm and the fruit tracking algorithm,and the algorithm was used to conduct a counting test on 10 orchard videos.The fitting correlation coefficient between algorithm counting and manual counting reached 0.9463,and a high counting accuracy of 94.6% was finally obtained.In summary,this paper proposes a feasible solution for fruit counting in tree rows,that is,an apple counting method based on deep learning and tree trunk tracking.Among them,the target detection model based on YOLOv4-tiny has high average detection accuracy,fast speed,and small model,which can achieve high-precision detection of fruits and tree trunks in videos in different environments.A tree trunk tracking method based on Euclidean distance matching provides reliable video motion displacement information for fruit matching counts.Through two-way matching based on Euclidean distance,removing abnormal matching and secondary matching scheme based on cross-union ratio,the connection between fruits is established to a greater extent.The video fruit counting algorithm developed by the integrated target detection algorithm and fruit tracking algorithm realizes the counting of fruits with high precision,provides new ideas and explores new ways for fruit yield measurement in orchards,so as to provide data support for the intelligent management of orchards and further promote apple industrialization,intelligence and automation.