Control And Implementation Of The Grabbing Of Pomelo Picking Harvestor Based On Monocular Vision

As a fruit for medicine and food,pomelo(Citrus maxima(Burm.)Merr.)is one of the bulk agricultural products.Usually,the pomelo orchard locates in hilly and mountainous areas,and the harvest work is totally manual work which means with low efficiency and high cost.To solve the problem of automatic picking and grasping of pomelo in mountain orchard,in this work,the requirements of field harvesting were analyzed and a control method of pomelo harvesting based on monocular vision was proposed.The target pomelo recognition,ranging,and positioning was achieved by using monocular camera,pomelo picking and grasping system based on three-axis actuator was established,equipment debugging and orchard picking and grasping experiments was completed,the feasibility of method was verified.The main research contents are as follows:(1)The recognition model for pomelo based on modified YOLOV5 S.According to the landform environment of orchard,monocular camera was used as visual system of harvesting operation,and YOLOV5 S deep learning model was selected for target recognition of pomelo based on the analysis of current research status of fruit and vegetable recognition.With 1104 pomelo images in different backgrounds as data sets,CBAM attention mechanism was added to YOLOV5 S model to improve the model’s recognition ability for small target pomelo and its anti-interference ability for irrelevant information.773 images were used as training set and 331 images were used as test set in the data set.The modified YOLOV5S-CBAM model performed with 95.33% accuracy,94.86% recall and 96.99% m AP.(2)The design and test of distance measurement algorithm for pomelo based on monocular vision.The mathematical relationship between pixel value of pomelo recognition frame and imaging distance was analyzed based on the YOLOV5S-CBAM model for identifying pomelo and the principle of monocular distance measurement.Combining with the existing mono-visual distance measurement model,with 13 different sized and shaped pomelos as the samples,17 groups of 271 images were shot with an interval of 4.75 cm within the distance range of 150cm-69.75 cm.16 groups of images were used to establish the mono-visual distance measurement model and one group of images was used to verify.The relative error of the 18 predicted distance values in the validation group was all less than 10%,of which the relative error of the 17 predicted values was less than 5%.The distance measurement requirements for pomelo in actual picking was basically satisfied by this model.(3)The design and validation of method for pomelo localization based on monocular vision.The target pomelo fruit localization method based on multiple frame images was selected,through Zhang Dingyou calibration method the camera’s internal and external parameters was obtained,and the camera position in the world coordinate system was fixed to(0,0,85cm);Based on the recognition and ranging of the target pomelo,combined with the pixel coordinates of the center point of the target pomelo recognition box,and based on the principle of coordinate system transformation,the three-dimensional position of the pomelo in the world coordinate system was obtained,thereby the relative position relationship between the target pomelo and the camera was determined.Within the imaging distance range of 50cm-150 cm,20 sets of pomelo fruit positioning experiments were conducted.When the distance value in the Z-axis direction was determined,the positioning deviation in the X-axis direction was less than 4.8cm,and the positioning deviation in the Y-axis direction was less than 4.5cm.(4)The design and implementation of visual servo picking action control based on position.The performance of the three-axis picking actuator was analyzed,the STM32 single chip microcomputer was selected as the motion controller of the mechanism,and the laser ranging sensor was selected as the motion stroke control and reset element of the mechanism,the overall circuit design and motion control of the mechanism was completed.The installation positions of the visual system and the end grabbing mechanism were determined based on the analysis of the picking and grasping work interval,and position based visual servo picking and grasping action was designed.The corresponding program design and communication design between the upper and lower computers were completed according to the picking and grasping action.In the upper computer program design,the program compilation of pomelo image acquisition,recognition,ranging,positioning,data transmission,and multi pomelos picking and grasping path optimization was implemented.In the lower computer program design,the program compilation of data reception,picking and grasping action execution,mechanism stroke control,and reset was implemented.(5)The debugging and on-site testing of pomelo picking and grasping system.The design and construction of the picking and grasping system was completed,the picking and grasping execution function and communication function of the system was tested based on upper and lower computer program control,and the picking and grasping system’s operational ability on site in the orchard was verified.The results of picking and grasping experiments on 20 groups of pomelo fruits showed that the picking and grasping system can effectively distinguish pomelo outside the picking and grasping work range,the success rate of picking and grasping was 66.2%,and the average working time for picking and grasping was 31.18 seconds,meeting the actual picking and grasping operation requirements of the orchard.This work provided a technical basis for the subsequent research on pomelo automatic picking equipment.