Counting Of Cherry Tomatoes Based On Deep Learning And Multi-objective Tracking

Accurate counting is the key to intelligent yield estimation of fruit and vegetables.Using computer vision technology to detect and count fruit targets is the research hotspot.In natural environment,fruit has overlap and occlusion problems,especially cherry tomatoes and strawberries,so it is difficult to detect fruit targets.At the same time,the canopy structure of plants is complex,and some fruit is not visible in a single view,so it is difficult to achieve accurate counting.This paper takes cherry tomatoes which planted in greenhouse as the research object,and carried out the research on cherry tomatoes fruit counting method based on deep learning and multi-object tracking.The main research contents and conclusions of this paper are as follows:(1)Establish cherry tomatoes datasets.According to the planting methods and plant characteristics of cherry tomatoes,white cloth is placed between the rows of plants to obtain video data of unilateral plants,and image data is obtained through frame extraction processing.Easy DL intelligent platform is applied to realize semi-automatic labeling,and three kinds of datasets are established,named single fruit detection dataset,fruit cluster detection dataset and fruit cluster count dataset.To solve the problem of unbalanced samples in fruit cluster count dataset,image synthesis was proposed to enhance some fruit cluster samples.(2)A counting method based on fruit cluster is proposed.To solve the problem of overlap and occlusion of fruit targets,the dense single fruit targets are transformed into sparse fruit cluster targets.YOLOv3 is used to detect the fruit cluster targets,and the regression method based on Res Net50 is applied to count the fruits of the fruit cluster,so as to realize the estimation of the total amount of fruit.Compared with the single fruit counting method,this method alleviates the problems of overlap and occlusion,and outputs the counting value of fruit cluster directly by the regression method,which simplifies the counting task effectively and realizes the cherry tomatoes counting at the image level well.(3)A video counting method based on fruit cluster detection and tracking is proposed.To address the problem that fruit is not visible in a single view,the multi-view information of cherry tomato plants is obtained by video,and the fruit cluster targets are detected and tracked by YOLOv3 and Deep Sort.The fruit cluster trajectory is predicted by the fruit cluster regression counting model based on Res Net50,and discusses the influence of different fruit cluster track counting methods on the accuracy of fruit counting.The results show that the most value is the best for fruit counting.The proposed method reflects the fruit shape from multiple perspectives and realizes the tracking of fruit cluster targets,which effectively overcome the problems of fruit overlap,occlusion and part of fruit invisibility,achieving the accurate counting of cherry tomatoes.