Research On Visual Precision Perception Algorithm For Intelligent Apple Picking Robot

At present,apple picking in China mainly relies on the manual picking method.In order to ensure timely harvesting of apples,a large number of laborers are required to participate in picking,but the number of laborers who can participate in picking in China’s rural areas is decreasing year by year,and apple picking will be severely constrained by labor shortage in the future.Therefore,it is important to study intelligent apple picking robots to alleviate the problem of harvesting labor shortage.Intelligent apple picking robots need to realize the work of sensing multiple types of information about apples and their surrounding scenes when operating.During the growth of apples,they are affected by the mutual shading between leaves,branches and fruits.Accurately determining the type of occlusion and the pickability of apples are both key technical challenges for apple perception.In addition,in order to improve the humanoid picking capability of intelligent apple picking robots,it is necessary to study the perception methods for apple posture.Finally,in order to achieve the obstacle avoidance perception capability of the intelligent apple picking robot during the picking process,it is necessary to study the accurate perception methods for tree branches and trunks.In this paper,the key perception requirements of the intelligent apple picking robot for fruit and surrounding scenes are studied in depth,mainly as follows:(1)To address the problem that the number of inter-class differences in the apple training dataset is significant,causing the network model to fail to achieve a balanced representation of performance on all types of targets,this study proposes a data augmentation method with high intra-class differences in apple detection.We used five popular lightweight object detection network models: yolox-s,yolov5 s,yolov4-s,yolov3-tiny,and effidentdet-d0 for validation experiments.The results show that the average detection accuracy of the five network models is improved to different degrees using the proposed data augmentation method.Specifically,the precision of yolox-s improved from 0.894 to 0.974,the recall improved from 0.845 to 0.972,and the mean Average Precision(m AP)improved from 0.892 to 0.919,which proved that the proposed data augmentation method has This demonstrates that the proposed data augmentation method has great potential for different shading apple detection tasks in orchard applications.(2)For the demand of pickable fruit detection by intelligent apple picking robots,this study proposes an improved pickable apple detection method based on yolov5 s.First,the yolov5 s teacher network model is obtained using fully supervised learning;then,the self-supervised learning training of the model backbone is conducted under the guidance of the teacher network model,and a network model backbone with better perception of image features is obtained;after that,the yolov5 s based on the selectable kernel attention mechanism is constructed,and the original backbone is replaced by this backbone,and the improved yolov5 s is obtained by fine-tuning the training.The results show that the improved network model achieves an average detection accuracy of 94.78%,93.86%,and 94.98% for pickable apple targets,intermediate apple targets,and non-pickable apple targets,respectively,on the test set,which is 3.23%,1.66%,and 4.85% better than the yolov5 s before the improvement.(3)In order to meet the needs of intelligent apple picking robots for accurate apple growth direction detection and accurate fruit segmentation and localization,the corresponding improved algorithm is proposed in this study.Firstly,the improved Openpose apple growth direction detection algorithm is proposed.The improved Openpose runs 6.56 times faster than before the improvement,and the average precision(Average precision 0.75,AP75)parameters at m AP and intersection ratio greater than 0.75 are increased by 9.18% and 6.35%,respectively.Then an improved UNet++ apple pixel precision segmentation algorithm is proposed.The results show that the improved UNet++ outperforms the original UNet++,UNet and Deep Lab V3+ network models in pixel-level segmentation metrics and edge quality accuracy metrics across the board,acquiring better metrics performance,reaching 94.08%,94.73%,95.87% and 90.91%.(4)To address the need of perceiving branch and trunk-like obstacles in the operation of intelligent apple picking robot man,this study proposes a lightweight branch and trunk segmentation network model based on Bisenet V2 improvement.This network model can achieve branch and trunk segmentation of fruit trees accurately in real time.The improved network model achieves 82.85%,77.57% and 87.36% in Acc,IOU and F1 metrics,respectively.The method proposed in this paper can realize the needs of intelligent apple picking robots for multiple types of information perception,which helps intelligent apple picking robots achieve accurate fruit and branch information extraction and provide safe operation for apple picking robots.