| With the development of economy, labor force transfering from rural to urban results to the shortage of labor and the rising of agricultural production cost. Citrus’ harvesting cost has reached to 35-50% of the total production cost. In order to reduce production cost, improve product’s competitiveness, and satisfy the demand of fresh fruit market, intelligent citrus picking robot (CPR) is one of the effective solutions. Due to the complexity of natural scenes, there are many difficulties in dealing with the recognition and location of the fruits and branches, as well as the safety of the manipulator’s operation. Hence there is still a long way from lab to practical application. In this paper, citrus and branches were identified using color information from natural scenes, and the relevant features were extracted. The fruit tree model was reconstructed using virtual reality technology based on the space information of the fruits and branches, obtained by binocular stereo vision system. The robot’s path planning was studied in the scene model composed of robot and fruit tree. The main achievements are summarized as follows:Self-adaptive segmentation of citrus and branch in natural scenes. The scene images were captured in different conditions. Through analyzing component images and fusing images in various color spaces, there were two distinct peaks (represented fruits and background respectively) in the histogram of G-B image, and Otsu could effectively segment the citrus regions. Compared the variation of RGB values among branches, fruits, leaves, and other regions, the two-class support vector machine (SVM) classifier and the multi-class SVM classifier were constructed to segment the branch regions or the branch regions and citrus regions simultaneously by using RGB values as input vector. At last, considering segmentational effects, computing time and other factors, Otsu based on G-B image was used to segment the fruit regions, and the two-class SVM classifier was used to segment the branch regions in this paper.Segmentation of branch using minimum noise fraction (MNF) combined with spectral angle mapper (SAM) based on multi-spectral image. Multi-band images of the same scene were captured using a self-made multi-spectral imaging system. Among these images, six component images were extracted to compose a multi-image. And then the top 4 principal images were extracted from it using MNF. Finally, the branch regions were segmented by SAM. It was proved that this method could efficiently segment branch regions and provide a new idea of object recognition for picking robot.Features extraction of citrus and branch in natural scenes. The features of citrus with different occluded degrees were extracted using cluster barycenter (CB), edge barycenter (EB), circular Hough transform (CHT), and least square circle fit (LSCF) respectively. By comparing the results, it was found that the robustness of CHT and LSCF were better than CB and EB’s. For the overlapped and occluded fruits, the watershed algorithm based on distance image was used to segment the citrus one by one. The occluded regions of citrus were restored by convex hull. Then the features were extracted from the restored contour by CHT and LSCF respectively. CHT took more time than LSCF for its great calculation. So, LSCF was used to extract the center coordinate and radius of citrus for further research. The skeleton of branches extracted by thinning was fit using line segments approximation. The feature point coordinates were extracted, and the radiuses of the feature points were obtained from the distance image of branches.Three-dimension information extraction of citrus and branch by binocular stereo vision. The citrus were matched based on center coordinates, and their space coordinates and actual radiuses were calculated by triangular measurement principle. The experiment results showed that the error of location was less than 10 mm in the robot’s working space (the distance from camera was 1.0-1.5 m). The disparity image was obtained using the matching based on region. The space coordinates and radiuses of branches’ feature points were calculated using triangular measurement principle from the disparity image.Real-time reconstruction of fruit tree. Through analyzing the real-time performance and transverse precision of truncated cone module, cylinder module, and multi-ladder cylinder module, the 4-ladder-12-prism method was applied to construct branch module. At first, a branch module was reconstructed at coordinate origin, and then located with correct posture after two rotations and one translation transformations. The tree model was generated by combining all the modules.Real-time path planning for CPR manipulator operation. The single-query, bi-directional, lazy collision checking, probabilistic roadmap method (SBL-PRM) was used to plan the path for the manipulator in scene model. The impact of maximum number of milestones S, neighboring threshold p, local path checking threshold s and path smoother steps N on time and success rate of planning were analyzed in the simulation experiments of two cases of picking non-occluded and occluded citrus. The results showed that SBL-PRM was effective in CPR real-time path planning. Binocular stereo vision was applied to identify and locate the citrus and branch in natural scenes. Non-collision path was planned for the manipulator in CPR working scene model reconstructed by vitual reality. Theoretical basis and technical support for real-time, secure, non-destructive picking of CPR were provided by this study. |
PDF Full Text Request