Research And Development Of Apple Picking Aystem Integrating Deep Learning Object Detection

Apples are the main fruit produced in China,and the production accounts for a quarter of the total fruit production in China.In the apple production process,apple picking has the characteristics of strong seasonality and short cycle.Due to the complex environment of orchards,domestic apple picking mainly relies on manual work.With the process of urbanization,the rural labor force has been greatly reduced and the labor cost has increased.Therefore,it is of great significance for the apple industry to adopt mechanical automation technology to solve the problem of apple picking by intelligent and mechanized means.In this study,we designed an apple picking system based on KINOVA six-degree-of-freedom robotic arm and depth camera for the problems of low apple recognition accuracy and low apple picking efficiency in natural environment,and the main research contents and works are as follows:(1)Improved YOLOv5s-based apple target detection.In order to accurately detect apple targets in orchards,an improved YOLOv5 s apple recognition model is proposed.Firstly,apple images are collected in orchards and Label Img is used for annotation to build the dataset.Then the YOLOv5 model and the detection principle are analyzed,and the lighter YOLOv5 s model is chosen as the base network.The asymmetric convolution block is added to enhance the feature extraction ability,the depth separable convolution is used to increase the accuracy while reducing the size of network parameters,the ECA attention module is added to enhance the learning ability of the network,and the Focal loss loss is used for the loss function to reduce the influence of negative samples on the results during the training process.The experimental results show that the improved network has an average detection accuracy of 96.6% on the workstation,which is 4.5 percentage points higher than the average accuracy of the original YOLOv5 s network,the detection network proposed in this study has a high detection accuracy.(2)Research on fruit localization based on depth cameras.According to the working principle of different depth cameras and the actual picking environment,the Azure Kinect DK depth camera is selected.The camera imaging model is analyzed,and the camera is calibrated using the Zhang Zhengyou calibration method,and the internal reference matrix of the camera is obtained.The three-dimensional positioning of the apple is realized based on the depth map,target detection results,and coordinate conversion.The localization method is tested,and the average localization error is 1.4167 mm.The types of hand-eye calibration and their advantages and disadvantages are analyzed,the eye outside the hand is selected as the mounting method between the camera and the robotic arm,the conversion relationship between the camera coordinate system and the robotic arm base coordinate system is analyzed,the hand-eye calibration test is conducted based on the Tsai method,and the hand-eye calibration matrix was solved.(3)The research of trajectory optimization of picking robotic arm based on particle swarm algorithm,taking KINOVA six-degree-of-freedom robotic arm as the research object,the construction of its linkage coordinate system based on D-H parameter method,the establishment of kinematic model and forward and inverse kinematic analysis,the analysis of working space of picking robotic arm according to Monte Carlo method,and the determination of picking range.A 3-5-3 polynomial is constructed to plan the trajectory of the robotic arm.In order to improve the performance of the picking robot arm,a particle swarm algorithm is used to optimize the trajectory generated by the 3-5-3 polynomial with time as the goal,and the total interpolation time is 3.0185 s after optimization.(4)The picking system is built and the picking experiment is completed.The software design of the picking system is completed under the ROS framework to realize distributed control.The test scenario is built,combined with the vision system and the robot arm control system to complete the picking.Randomly fixing the apple position,several groups of picking tests are conducted,and the picking success rate is 74%,and the average picking time is 16.172 s,which verified the effectiveness and stability of the system.