Picking Behavior Of Grape Harvesting Robot Based On Visual Perception And Its Virtual Experiment

With the advancement of Chinese agricultural modernization,fruit planting becomes more standardized,clustering and industrialized.To improve the picking efficiency,ensure the fruit quality and reduce labor intensity,it is urgently needed to develop a high-efficient picking robot.In the development process of picking robot,visual detection on picking target,positioning on picking point,calculation on anti-collision bounding volume and planning on picking behavior are important for picking robot to be practical and commercialized.While the traditional experiment on visual positioning and picking behavior are usually carried out in the orchard.It is easily to be affected by the harvest season of picking objects,weather condition and the picking venue.The developed algorithm couldn't be verified timely and efficiently.This extend the prototype development cycle.In this paper,taking the greenhouse frame grape cultivation as the research object,conduct in-depth research on the multi-dimensional visual perception behavior(picking point,anti-collision bounding volume)of picking robot and virtual test.It provided technical support and method theory for precise anti-collision picking,thus accelerate and promote the mechanized,automated and intelligent process of Chinese fruit picking.Firstly,referring to the related literature at home and abroad about visual perception,positioning,and anti-collision picking behavior of picking robot,this paper takes depth analysis on fruit planting standard,fruit shape,color and texture features.Then,based on the method and theory of intelligent computing,machine learning,stereo vision,robot behavior planning and virtual reality,by integrating use of hardware in the loop simulation and prototype test,took depth research on core contents about target image recognition,picking point positioning on fruit stalk,anti-collision bounding volume calculation and picking behavior virtual test.The details are as follows:(1)To solve the problem of grape image recognition in orchard environment,a method of grape image segmentation based on artificial bee colony optimization fuzzy clustering was proposed firstly.By using the nature behavior of bees,this method optimized the traditional fuzzy clustering algorithm which is easy to fall into local optimum;With the collaboration of bee colony,follow bees and computerized bee,the optimal clustering center of grape image could be solved and the image segmentation are realized.Secondly,design an Adaboost-MutipleColor grape recognition algorithm by combining the ensemble learning and multi-color space.This algorithm constructed 4 weak classifiers by highlighting the grape color space to most;obtained strong classifier by use of Adaboost framework and integrated training on 4 weak classifiers,then classified the target and background by strong classifier,and realized the target recognition finally.The above two algorithms were tested by using the image of summer black grapes collected in orchard environment.The results showed that the recognition rate of these two methods is up to 90.33% and 93.74% respectively.(2)By analyzing morphology of grape stems and growth posture of grape under the action of gravity,designed a picking point positioning method for a signal cluster grape based on image segmentation and minimum distance between dot and line.Testing on 300 grape images collected from orchard,the accuracy of picking point could reach up to 88.33%.In addition,in order to recognize and position the overlapped and adhering grapes under orchard environment accurately,a target recognition and extraction on two overlapped and adhering grapes clusters were proposed based on contour analysis.This method extracted target of each cluster respectively by solving the inflection point of folded edge at the junction,and then calculating the picking point of two overlapped and adhering grape cluster by positioning the picking point of single grape cluster.Testing on 27 images from overlapped and adhering grapes,positioning success rate was up to 81.48%.(3)To facilitate an intelligent anti-collision picking behavior planning for a picking robot,a new calculation and localization method for grape bounding volume based on binocular stereo vision was proposed.Firstly,obtained the picking point on grape stems and berry center by visual inspection;then calculated 3D coordinates of picking point and grape fruit by using stereo matching and triangulation principle;constructed grape space coordinates by taking space coordinates of picking point as base point and solved the maximum section of grape;finally rotated the section around the central axis and scanned to get the grape bounding volume.The proposed method was tested by binocular vision images collected at different distances,and the experimental results were analyzed from the following three aspects: the positioning error,the accuracy of the bounding volume and the real-time performance.The results showed that when the depth distance is within 1000 mm,the height error of the surrounding body is less than 4.95%,the maximum diameter error is less than 5.64%,and the running time of the algorithm is between 0.38 and 0.69 s.(4)To verify and test the robot vision positioning and behavior control algorithm,a set of virtual test system based on hardware in loop simulation was designed and developed.Firstly,the system obtained space coordinates of grape picking point and anti-collision bounding volume through physical visual system.Then used the picking robot developed by our own laboratory as prototype,developed the virtual robot simulation system.Finally,virtual robot carried behavior planning about anti-collision path planning and inverse kinematics behavior solution by picking point coordinates and anti-collision bounding volume,then control the end effector to run to the picking point and executed picking.34 time virtual experiments were performed in the hardware in the loop system in which successful visual positioning tests are 29 times(85.29%),successful path planning 28 times(82.35%)and successful clamping and cutting 28 times(82.35%).The results showed that this virtual test system can test and verify the algorithm for visual positioning,robot path planning and clamping and cutting behavior.(5)Integrated the core algorithms about visual recognition and positioning and picking behavior studied in(1)-(4)into software system.Then applied the related algorithms to 6 degree of freedom control picking robot developed by our own laboratory and performed test.It verified the effectiveness of the algorithm proposed in this paper.