Research On Recognition And Localization System Of Harvest Robot Based On Information Fusion

With the development of artificial intelligent and robot technology,automated and intelligent agricultural is development trend in the future.There is a complex apple orchard environment in China that the apple trees always have high fruit filling rate and apples on the tree have different attitudes,large diameters with high overlapping and occlusion rate.The images collected by the visual system are mostly overlapping images,so the current fruit visual technology cannot be fully applied to the recognition and localization requirements of the northern apple orchard.In view of the above characteristics of the northern apple orchard scene,this paper has carried out detailed research on the apple visual recognition and localization system.The main research contents are as follows:(1)Establish a stereo vision recognition system for apple harvest robot.The internal Kinect device is calibrated to obtain internal parameters and external parameters.The rigid registration method is used to calculate the mapping matrix of the depth map and the color map,and then the color and depth information are combined to obtain the apple eigenvector P[x,y,z,r,g,b].(2)For the purpose of recognizing the apples in the orchards,A method to extract the apple grayscale feature is proposed.Due to the complex environment of apple orchards,single gray scale method cannot get good results.So in this paper,4 kinds of commonly used method of gray scale include R-G,ExR,S component are compared and analyzed,using otsu global optimization threshold and k-means clustering for grayscale features.The apple is roughly segmented from the background region.the suppression effect of background noise and apple feature extraction in different environments are analyzed,and the time complexity and feature extraction effect are compared,and the gray operator and image required for color feature extraction of apple picture are determined.Segmentation algorithm to extract rough segmentation image of target fruit.(3)The method of fruit gesture recognition was studied.The second-order moment method,the contour and centroid distance shortest method and the slope variance method are used to identify the fruit axis direction of the single-fruit apple target with the contour visible.The fruit axis recognition effect of each method is analyzed and compared,and the second-order moment method is not applicable.For the problem of multiple symmetry axes,an improved secondorder moment method for the maximum distance of parallel tangents is proposed,and the multimethod decision fusion method is used to identify the fruit posture of single-fruit apples with visible contours.(4)A fine segmentation method suitable for large-area overlapping apples is proposed.For the overlapping apple target,the depth information in the apple eigenvector P[x,y,z,r,g,b] is extracted,the apple characteristic feature probability distribution is counted,and the segmentation of the overlapping apple is completed by the adaptive threshold region growing segmentation method.Aiming at the complex environment of apple orchard in northern China,this paper designs a stereoscopic visual recognition system for harvest robots.The research and experiments show that the system has good adaptability.The fruit attitude recognition algorithm proposed in this study can provide a theoretical basis for the accurate identification of the fruit axis direction.The overlapping fruit recognition algorithm proposed in this study can accurately identify the three-dimensional coordinates of the fruit,which provides a theoretical basis for accurate positioning of the harvest robot fruit.