Research On Rapid Identification And Localization Of Navel Orange Fruits In Natural Environment

Gannan navel orange with the characteristics of fragrant,thick and sweet,and nutritious flesh,is one of the specialty fruits in Jiangxi province,which is deeply loved by consumers and widely planted in southern Jiangxi.But at present navel oranges are mainly picked by manual,labor intensity during the picking period.According to the statistics,data,picking cost accounts for 30%～40% of the total production cost.In this paper,computer vision technology is used to study the identification and positioning method of navel orange fruit in the natural environment,which can provide the research foundation for the research and development of navel orange picking robot,so as to realize the automation and intelligent picking of navel orange in southern Jiangxi.The main research contents are as follows:(1)Research on image acquisition of navel orange fruits in natural environment and pretreatment methods.According to the actual requirements of the project,Kinect V2 depth camera was used to collect the color images and depth images of navel orange fruit.Under circumstance of ensuring the accuracy and stability of the camera,navel orange fruit images with different levels of illumination and different shooting angles were collected to ensure the representativeness of recognition model.According to the color characteristics of navel orange fruits in the natural environment,navel orange fruit images and the component images in RGB,HSV and Lab were made,and compared with each other.The results showed that there was a great difference between the color image of navel orange fruits and the background image in RGB color space.Finally,RGB color space was selected as the color space model of navel orange fruit color image samples in natural environment.In order to remove the interference of noise factors such as light in natural environment,median filtering,wavelet and fast guidance filtering were used to preprocess the raw images and the preprocessed results were compared.The experimental results showed that the fast guidance filtering algorithm could better remove the original image noisr information,and obviously improve the contrast of navel orange image and highlight the characteristic information.(2)Establishment and improvement of navel orange fruit recognition model.YOLO V3 target recognition algorithm was used to establish the navel orange fruit recognition model and the improved algorithm was made based on YOLO V3.Darknet-53 with residual modules was used as feature extraction network,and the 3-scale detection network based on the multi-scale fusion was reduced to 2-scale detection network.GIo U boundary loss function was introduced to replace the original loss function,and DBSCAN+Kmeans clustering algorithm was used to analyze the training data and optimize the size of prior frames for prediction branches.Training was carried out by transfer learning method.Five test sets of single fruit,light direction,backlight,fruit overlap and branches and leaves occlusion were designed to investigate the improved YOLO V3 model and compared with the original YOLO V3 model,SSD model and Faster R-CNN model.The results showed that the comprehensive performance of the improved model was better than other networks in five environments,especially in the real planting environment,the recognition accuracy was 91.22%,the recall rate was 97.30%,the average F1 is 94.16%,and the recognition rate was about 26.48 FPS.(3)Nased navel orange fruit localization method research based on Kinect V2 depth camera.In order to realize the spatial positioning of navel orange fruit picking points,the conversion process and conversion formula between four coordinate systems were introduced.Based on camera calibration experiment,the internal and external parameters and correlation matrix of Kinect V2 camera were obtained by Zhang Zhengyou calibration method,and the reprojection error analysis of camera calibration results was conducted.The results showed that the average reprojection error was within the allowable error range and meet the practical application conditions.Based on the navel orange fruit recognition model established above,the 3d coordinates of picking points of navel orange fruits in the camera coordinate system were obtained by color map and depth map registration,and the 3D coordinate error experiment was designed.The experimental results showed that the positioning relative error of the improved YOLO V3 model combined with Kinect V2 sensor was within 2%.It can meet the requirements of threedimensional location of navel orange fruit picking points.(4)Development of the identification and positioning system for navel orange picking robot.Based on the identification model of navel orange fruit in the natural environment and the three-dimensional positioning method of picking points,the identification and positioning system of navel orange picking robot was developed.The system was based on Windows operating system,and Open CV computer vision library combined with Qt5.10 integrated development environment were used to develop the system.The system function modules mainly included: system login module,image import and preservation module,image pretreatment module,navel orange fruit recognition module,fruit spatial positioning module,etc.The developed system can be used to achieve navel orange fruit recognition and positioning.