Visual Recognition And Guidance Of Tea Cutting Position For Autonomous Tea Picking Robot

At present,the scale of the tea market is increasing,and the market demand for famous and excellent tea is also increasing.The picking of famous and excellent teas is still manual picking,which requires tea farmers to have rich picking experience,so the cost is relatively high and the efficiency is relatively low.Traditional tea picking mechanisms cannot distinguish tenders or young leaves from old leaves,and the quality of tea is easily damaged during the picking process,which is not suitable for picking famous and excellent teas.Therefore,the intelligentization and automation of tea picking machinery is very important.The purpose of this paper is to build an autonomous tea picking robot platform,identify and locate tea picking points,and achieve highprecision and high-efficiency robotic autonomous picking operations.This paper builds the hardware platform and software platform of autonomous tea picking robot.The hardware mechanism mainly includes a biped crawler mobile robot,a SCARA machine arm,a CMOS image sensor and a tea shearer.Aiming at this hardware platform,we have designed and implemented a visual guidance software system,which implements functions such as robot communication,visual system calibration and image acquisition,3D reconstruction,point cloud segmentation,picking point calculation,visual tracking,and robotic arm guidance control.These functions provide a solid basis for the automation of picking operations.This article analyzes the movement characteristics of the SCARA manipulator,and proposes the general procedures of the SCARA manipulator tool center(TCP)calibration,camera calibration and hand-eye calibration.Since the SCARA robot has only one degree of freedom of rotation,that is,it can only rotate around the Z axis,the calibration of hand-eye matrix of the SCARA robot is divided into two parts.The first part is solved by the matrix direct product method,and the second part is solved by using the coordinate system transformation relationship to obtain the Z-direction translation component of the hand-eye matrix coupled with the Z-direction translation of TCP together.The final accuracy of visual guidance is closely related to the accuracy of the hand-eye calibration of the robotic arm,so this article focuses on the optimal hand-eye calibration of the SCARA robotic arm.The algorithm minimizes the infinite norm of the angle error vector,and searches recursively in the parameter space through the branch and bound method.The previous optimal hand-eye calibration algorithm for general robot systems searches over a 3-dimensional rotating space,which requires a large amount of calculation.This paper uses the special structural properties of the SCARA manipulator arm and uses the knowledge of Lie group to couple the hand-eye matrix parameters to define a new two-dimensional parameter space.Therefore,the algorithm proposed in this paper greatly reduces the calculation time.Compared to the comparison algorithm which takes 2326.3s,the calculation time of this algorithm is 12.3s.Considering the difficulty of image segmentation of natural scenarios,this paper proposes an algorithm to segment individual plant leaves using depth information.The algorithm first performs stereo correction through quasiEuclidean reconstruction,and then uses the sum of absolute difference(SAD)and the zero-mean normalized cross-correlation coefficient(ZNCC)to fuse the two cost functions through confidence,and then further fuse with other sample points within a same superpixel.The multi-depth hypothesis is generated and finally a suitable input point cloud is obtained.This part only takes about 1.2s.Subsequently,Tensor Voting is used to denoise the point cloud and estimate the manifold distance of the data point.DBSCAN clustering algorithm based on the new-defined manifold distance is used to segment input point clouds and obtain individual leaf point cloud.The simulation data and real plant images show that the algorithm can segment each single leaf in a complex natural environment.On the basis of the point cloud segmentation results,this paper proposes an algorithm for accurately calculating the picked points by minimizing the distance function.Finally,this paper proposes a vision guidance algorithm for closed-loop control strategies.The stereo matching algorithm using sub-window segmentation and adaptive weighting can find the corresponding picking point on the image to be matched more robustly,thereby the initial three-dimensional coordinates of the picking point can be reconstructed.By using extended Kalman filter and particle filter,and real-time tracking the projection point of the picking point,the spatial position of the picking point could be estimate dynamically.The actual experiment shows that the visual guidance control algorithm based on the extended Kalman filter can reduce the error by about42.3%.In this paper,aiming at the tea picking operation of the autonomous tea picking robot,the visual recognition and guidance system of the autonomous tea picking robot’s tea cutting position is constructed,which solves the problems of the difficult visual identification and low positioning accuracy of the tea cutting position,and realizes the high-precision automatic tea picking function.