Research On Fruit Recognition And Location Based On Deep Learning And RGB-D Depth Information

Fruit recognition and location is a key component of the robot harvest vision system,and it is also an important research field of computer vision.Traditional identification algorithms have always been limited by occlusion,illumination,and shape differences,which make the recognition and location of fruits meet many difficulties.The lack of robustness of algorithms in non-structural environments has become a major bottleneck restricting the development of harvest robots.With the continuous development of recognition technology,convolutional neural networks have the advantages of strong ability of expressing non-linear features,good generalization performance,etc.,and can well complete the task of recognition and detection of objects.It makes the deep learning detection technology available for the realization of automatic harvest.As an important part of realizing fruit automatic harvest,this paper mainly used as a background to study and explore the detection and location techniques of multiple fruits,and focuses on the detection and positioning scheme of four kinds of fruits,such as navel orange,litchi,citrus and apple in unstructured environment,to improve the overall accuracy and robustness of the fruit detection and positioning system.This paper makes use of the theory of computer vision and deep learning to deeply research and analyze the identification and positioning of multiple fruit.The following work was carried out around the subject of this paper:(1)Traditional recognition and deep learning methods are applied to study the fruit recognition and detection in unstructured environments,respectively.By analyzing the color model of the fruit,the Cr component based on the YCrCb color model is selected as the object of the OTSU threshold segmentation algorithm,the results showed that the integrated F1 value was 61.45% under different illumination.Aiming at the problem that Otsu method can't satisfy target detection in all kinds of unstructured environments,a fruit target detection model based on SSD method is proposed.By comparison of different illumination angles and occlusion ratios for fruit images in unstructured environments,the results show that when the size of the input image is 300×300,the F1 value of illumination and occlusion below 75% is 95.18% and 90.74%.When the input image size is 512×512,the F1 value can reach 95.78% and 92.04%,respectively.(2)In this paper,a fruit image depth detection framework based on residual learning model and feature Pyramid information fusion is proposed to solve the problem of repeated multi-layer feature extraction and low target recognition accuracy in SSD detection framework.Through data enhancement operations,it is verified that a rich data set can effectively improve the robustness of the algorithm.Compared with the popular detection frameworks of Faster R-CNN and R-FCN,the experimental results show that the mean average precision of the ResNet-101 and feature fusion network models for fruit image recognition detection is 88.4% and 86.81%,respectively.(3)The paper proposes a fruit harvest visual recognition and positioning scheme based on RGB-D camera depth ranging.It compensates the error of the camera's depth ranging error analysis,and extracts the camera's color and depth maps for registration to obtain threedimensional information of the fruit target area.The experimental results show that the accuracy of the depth measurement of the fruit within 2cm,which can meet the requirements of the harvest task.