Research On Apple Picking Of Robot Arm Based On Deep Learning

In agriculture,the research and development of automatic harvesting robot technology is the key for the industrialization and transformation of China's agricultural.This article focuses on automated apple picking task in real scenarios.It has important significance and practical value for the development of intelligent agricultural equipment.Combined with deep learning methods,this study uses RGB-D cameras to obtain the information of scenarios to achieve the detection of apple in images and the segmentation of apple 3D point clouds.Path planning and simulation.Specifically,this study completed the following work:Firstly,data collection and making dataset.This study collected apple RGB-D data including four varieties of apple trees and two lighting conditions in four orchards in Beijing and Shandong province.There are 5660 RGB images in the dataset,and the depth data of scenes corresponding to the RGB images were collected at the same time.All data were manually labeled to make RGB-D dataset,which was used to train optimized deep convolutional neural networks.Secondly,combining the regions of interest(ROIs)from RGB images with the RGB-D camera mathematical model,the apple point clouds were segmented.Moreover,the overlap rates of the segmented point clouds for four varieties of apples are more than 96%.Thirdly,a contact-based hand-eye calibration method was proposed for the Eye-To-Hand vision system,which realized the hand-eye calibration of RGB-D camera and robot by solving the transformation relationship between coordinate systems.Finally,path planning was proposed for multi-apple picking.In addition,based on the ant colony algorithm,a decision mechanism for calculating the global optimal path of the robot arm was proposed.According to the principle of the shortest global path,the multi-apple picking path was planned.The conclusions of this study have theoretical reference significance for such issues as apple detection,apple spatial positioning analysis and picking path planning in apple automatic picking task.