Grape Clusters Tracking And Counting Method Based On Machine Vision

Accurate fruit counts will help the wine industry make better pre-harvest logistics and decisions and produce higher quality wines,critical to improving winery efficiency and wine marketing strategies.To address the current method of estimating grape orchard productivity relying on manual sampling,which is destructive and time-consuming,among other problems.This study takes wine grapes in the field as the object,establishes a grape berry recognition model based on machine vision and deep learning algorithms in a complex background,and develops a corresponding tracking and counting algorithm to realize automatic tracking and counting of grape clusters from video data,with a view to providing technical support for intelligent management decisions in grape orchards.The main research contents and corresponding conclusions of this study are as follows.（1）Grape clusters instance segmentation method based on Mask R-CNN.To address the problems of low detection and segmentation accuracy and poor model generalization performance due to the huge variation of grape clusters shape,leaf shading,trunk shading and grape overlapping,this study proposes a new backbone network feature extraction network,Res Net50-FPN-ED,to improve the Mask R-CNN instance segmentation model.Firstly,an efficient channel attention mechanism ECA module was introduced in the backbone feature network to correct the extracted features for more accurate detection of grape clusters.In the feature pyramid fusion stage,a richer high-dimensional image detail was obtained by training a set of dense upsampling convolution instead of the traditional nearest neighbor interpolation upsampling to improve the accuracy of the model segmentation.In addition,the generalization performance of the model was improved by training the model on two different datasets.The experimental results indicated that the improved Mask R-CNN grape clusters instance segmentation model achieved AP of 60.1%on the detection task,compared to 1.4%and 1.8%improvement over the original Mask R-CNN（Res Net50-FPN）and Faster R-CNN（Res Net50-FPN）.The AP reached 59.5%on the segmentation task,an improvement of 1.6%and 2.2%compared to the original Mask R-CNN and SOLOv2,respectively.When tested on different datasets,the improved model still has better detection and segmentation accuracy as well as inter-varietal generalization performance in complex growth environments,and can effectively identify grape clusters in complex backgrounds.（2）Rapid detection method of grape clusters based on channel pruning with YOLOv5s.To address the problem that the grape clusters recognition model in the field is limited by its model size and the number of parameters size by the high-performance detection platform and the poor real-time performance.In this study,the YOLOv5s model was pruned based on the channel pruning algorithm to obtain a more lightweight YOLOv5s grape berry detection model.After pruning,the amount of model parameters was reduced by 79%,the number of floating-point operations was reduced by 58%,the model size was reduced by 11.1 MB（76%）,and the pruned model was only 3.3 MB,and the inference time of the model was reduced by 7×10^-4s.In addition,Soft-NMS was introduced in the prediction stage to improve the detection performance of the overlapping grape clusters.The results on the image test set showed that the m AP of the fine-tuned model reached 82.3%and the F1 score reached 79.5%,which effectively reduced the parameters and complexity of the model while ensuring the detection accuracy.To verify the effectiveness of the proposed method,the performance of the proposed method was compared with four object detection algorithms,YOLOv4,YOLOv4-tiny,SSD300,and Efficient Det-D1,under the same test set,and the recall,F1-score,and m AP of the proposed method were always higher than the other four algorithms,and the model size was the smallest.The recall of the proposed method is 11.08%,15.87%,26.54%,and 6.29%higher than YOLOv4,YOLOv4-tiny,SSD 300,and Efficient Det-D1,respectively.F1-score is 5.5%,10.5%,16.5%,and 2.5%higher,respectively.m AP is 2.19%,9.43%,18.52%,and 1.11%higher,respectively.The detection speed is 124.1 f/s,38.8 f/s,and 138.3 f/s higher than YOLOv4,SSD 300,and Efficient Det-D1,respectively,indicating that the channel pruning-based algorithm is effective for light weighting YOLOv5s.（3）Grape clusters tracking and counting method.Two clusters tracking counting methods were investigated for different application scenarios and practical needs.The improved Mask R-CNN grape clusters instance segmentation model combined with the SORT tracking algorithm achieves accurate tracking of berries at the pixel level and facilitates the expansion of more complex and accurate grape phenotype analysis.The pruned YOLOv5s lightweight cluster detection model was used as a detector combined with the SORT algorithm for real-time online fast tracking of grape clusters in the field.To solve the problem of low counting accuracy due to ID switching during tracking,counting accuracy was improved by introducing counting lines in the video stream data,and the effect of the position of the counting lines on the counting results was discussed.According to the vehicle’s moving direction,two counting modes were set up to realize the tracking and counting of clusters.In addition,two grape clusters tracking counting algorithms were analyzed and compared.The test results on 8videos showed that the improved Mask R-CNN and the pruned YOLOv5s model combined with the SORT tracking algorithm achieved an average counting accuracy of 89.17%and84.9%,respectively,and the processing speed could reach up to 6.8f/s and 50.4f/s,respectively.The correlation coefficient R² between the counting results of the two algorithms and the manual counting reached up to 0.9905,which can meet the requirements of fruit counting in grape orchards under different application scenarios.