Research On The Technologies Of Target Recognition,Localization And Control For Tomato Harvesting Robot

As a typical intelligent agricultural equipment,the harvesting robot integrates various advanced technologies such as robotic,sensing and control.Because of the need to adapt to the complex agricultural scene,the development of harvesting robot is a high technical difficulty.At present,the major technical bottlenecks restricting the performance improvement of harvesting robot are the issues of target rcognition,location and control.In this paper,the tomato harvesting robot was used as reseach object,the technologies of target recognition,location and control for tomato harvesting robot were studied.These works involve the theoretical study,technical implementation,and validation test of various technology such as the design of tomato harvesting robot,image processing and machine learning.The main research contents and conclusions of this paper can be summarized as follows:(1)According to the environment characteristics of tomato greenhouse and task demand for robotic harvesting,the software and hardware of a tomato harvesting robot are designed and developed.A dual-arm robot,two kinds of end-effectors,and a mobile underpan of the tomato harvesting robot are designed respectively.According to the requirements of control performance,a robot driver and control system based on EtherCAT and a control system based on Arduino are also developed.The software system for the tomato harvesting robot which is developed based on ROS adoptes the design scheme of logical layering and functional modularization.It includes various driver layer programs and application layer programs which are oriented to the hardware and functional model respectively.(2)For overcoming the intfluence of environment perception for tomato harvesting robot in unstructured environment,a ripe tomato segmentation algorithm based on multiple feature images fusion is proposed.Two novel feature images,the a*-component image and the I-component image,are extracted from L*a*b* color space and YIQ color space at first.Then,wavelet transformation is adopted to fuse the two feature images at the pixel level.In order to segment the target tomato from the background,an adaptive threshold algorithm is used to get the optimal threshold.In the tests,93% target tomatoes were successed segmented of overall samples.Moreover,the results also indicated that the influence by illumination and overlapping was effectively reduced.Results also showed that the proposed feature images fusion method could improved the power of environment perception for tomato harvesting robot.(3)For meeting the challenges of high recognition accuracy,high robustness and rapidity,a ripe tomato recognition algorithm based on combination of AdaBoost classifier and color feature classifier is proposed.Through off-line training,an AdaBoost classifier for tomato recognition is trained.Then a color feature classifier based on average pixel value is also obtained through color analysis.By the mthod of cascade,the two classifiers are combined,which is applied to recognize the target tomato on-line.The conculsion indicated that the C type Haar-like feature was the best Haar-like feature used for recognizing the tomato.The combination of AdaBoost classification and colour analysis could correctly recognize over 95% of ripe tomatoes in the real-world environment.Meanwhile,the false negative rate was about 5%.The good robustness and rapidity of the proposed algorithm could be satisfied the requirements of tomato harvesting robots.(4)In order to realize the locating the target tomato and positioning the end-effectors,a method of control for harvesting tomato based on binocular locating is proposed.The binoclular camera is used for acquiring the point cloud data of tomato harvesting robot operation scene.The the three-dimensional virual environment of harvesting robot is built by using the point cloud data.The spatial location of target tomato is obtainded in the virtual environment.According to the result of target tomato location,manipulator motion trajectory planning for each joint motion parameters can be calculated.Finally,the robot completes harvesting motion by the control of obtained motion parameters.The accuracy of the binocular locating was about 2mm.The positioning errors of end-effecoters for tomato harvesting robot were distributed within 6mm-10 mm.The testing result showed that the propoed control mthod could be applied to the tomato harvesting robot for completing the robotic harvesting in the simulated conditions.(5)For eliminating the errors caused by binocular locating and manipulator motion,a method of micro-operation control for tomato harevesting robot is developed based on visual servo.Using eye-in-hand configuration,the visual servo control system is dopted IBVS approach.The relationship between image and cartesian space which is described as hand eye model can be identificated online by the Kalman filter.Finally,the visual servo controller based on PI control strategy for harvesting robot is designed.Through the field tesing,it was shown that the precise positioning of end-effector was completed in a small space under the control of designed visual servo controller.Moreover,the additional testing motion was not required by employing the proposed control method.Becased of the good self-adaption to working enviroment,it could ensure that the tomato harvesting robot completed the picking operation.