Research On Fruit Recognition And Positioning Based On Binocular Vision In Unstructured Field Environment

The palm fruit,durian,pineapple and other fruits grown in the complex environment of the natural field are manually picked and placed on the roadside in the field.There are problems of large collection workload and low secondary transfer efficiency.As a kind of fruit picking agricultural robot,it needs to adapt to complex agricultural operations.Environmental intelligent agricultural equipment can greatly improve work efficiency and help save the manual workload and cost of fruit picking.At present,how to accurately identify and locate fruits in the field is one of the key technologies of fruit picking agricultural robots.This article focuses on the spatial positioning of the fruits to be picked in the complex environment of the field,focusing on the following aspects:Data collection is performed on pineapple fruit images under different lighting conditions,and the image is filtered and preprocessed to remove part of the noise while highlighting the details of the fruit area,and through geometric transformation and image color changes for data enhancement and data amplification.At the same time,it analyzes the shortcomings of the existing algorithms for the feature extraction and recognition methods of fruits,and considers to propose an improved YOLOv3 deep learning target detection method.Using the Tensorflow deep learning framework,the lightweight network Mobile Net network is replaced by the Darknet-53 network in the YOLOv3 model.Structure,the residual network structure is used in the auxiliary convolutional layer to replace part of the convolutional network structure,serving as the basic structure of this layer to reduce the amount of convolution operation for extracting image features,and comparing different algorithms to find that the recognition accuracy is not reduced In the case of improving the recognition speed and stability of the network.For obtaining the spatial coordinate information of the fruit,the three basic principles of TOF camera,structured light camera,and binocular vision,which are currently mainstream for obtaining image depth information,are compared.Finally,the triangulation principle of the binocular depth camera Real Sense D435 i is adopted to collect the fruits to be picked.Image,and detailed introduction to the technical principles,hardware structure and camera parameters of the binocular depth camera Real Sense D435 i,and camera calibration.After the two-dimensional coordinates of the fruit are obtained through the improved YOLOv3 deep learning model,the SDK is developed based on the depth camera.The package obtains the corresponding depth information,performs coordinate system transformation to obtain the actual spatial positioning coordinates,and designs the fruit’s spatial positioning system.Aiming at the problem that the fruit’s spatial positioning system is more convenient to deploy on the agricultural robot side,the NVIDIA Jetson Xavier development version is used to connect the Real Sense D435 i binocular depth camera to the development board to collect and process image information,and to use the trained fruit on the PC side The target detection model is migrated to the development board,and the fruit target recognition and positioning effect is detected in real time by outputting the running results to the screen configured on the development board.It is found that the model in this paper is effective in identifying the fruits to be picked in the weed occlusion and overlapping scenes.The accuracy is high.The frame rate of video reading and detection for the binocular depth camera can reach up to about 24 FPS,while the frame rate of obtaining three-dimensional information of the fruit at the same time during detection is only about10 FPS,because the fruit depth map is not read.The calculated fruit space coordinates will reduce the point detection frame rate.The research on further optimizing the internal parameters and performance of the camera will be carried out later to provide a good solution for the automatic picking of agricultural robots in the field,and lay a foundation for the next development of agricultural robots.Better technical basis for image processing.