Localing Premium Tea Picking Points By Keypoint Detection Algorithm

With the continuous pursuit of tea quality by consumers,the production of premium tea has been increasing,and there are also higher requirements for their tea picking methods.Nowadays,the picking of buds of premium tea is mainly based on inefficient and costly manual tea picking.Therefore,it is urgent to use mechanization technology for the harvesting of premium tea varieties,and the first research is to be able to detect,identify and accurately locate the buds of premium tea and their picking points.Taking the buds of Longjing tea plant in the natural environment as the research object,using the object detection algorithm and keypoint detection algorithm in deep learning,and using the RGB-D camera as the color image and depth image acquisition,the relevant technical research on the two-dimensional recognition and three-dimensional positioning of tea bud picking points is completed,and the main research content is as follows:(1)In this research,the tea bud data were collected from natural tea gardens,and a dataset was constructed by considering the unique situation of dense tea buds,overlapping branches and leaves in the natural environment,as well as the brightness change under natural light.In addition,this study measured the model performance by introducing evaluation indicators in the COCO dataset,in which OKS was specifically used as the keypoint detection threshold to evaluate the keypoint detection results.(2)In this research,a developed Mask R-CNN network was built and trained to detect tea objects in images,and keypoint detection branches containing fully convolutional networks(FCN)were added to locate key picking points on tea objects.The convolutional neural network built according to this method obtained 88.6% precision and 90.3% recall rate for bud leaf target detection through dataset training.The average precision for keypoint targeting was 85.6% and the recall was 83.3%.Experiments showed that the developed algorithm has robustness for a variety of tea bud picking scenarios,which provided a possibility and theoretical basis for end-to-end picking point identification in the tea bud harvesting process.(3)In this research,a conversion algorithm from image coordinates to real physical space coordinates was established,and the position coordinates of the picking point detected by the key points are converted into the coordinates of the real physical space according to the camera calibration and hand-eye calibration results,and the three-dimensional positioning of the keypoints of the picking points of premium tea buds was completed.The positioning method was verified by building a hardware platform for positioning tea bud picking points including a depth camera and a robotic arm.Experiments showed that the established coordinate conversion method can well locate the coordinates of tea bud picking points in three-dimensional space.