Recognition And Positioning Of Ripe Tomatoes Based On Binocular Vision

Since 2017,China’s tomato planting area and yield have ranked first in the world,with an annual output of more than 55 million tons,accounting for about 7% of the total vegetable output.With the advancement of social industrialization and informatization,the disadvantages of traditional manual picking of poor quality and low efficiency have gradually emerged.In recent years,people’s requirements for the quality of fresh fruits and vegetables have become higher and higher,and with the increasingly serious problem of my country’s population aging,traditional agriculture is facing the problem of insufficient labor resources.This subject is based on binocular stereo vision and deep learning algorithms to study the identification and positioning of mature tomatoes,which is of great significance for the realization of rapid automatic picking of tomatoes.In view of the actual greenhouse application environmental conditions,the machine vision tomato picking process is subject to the conversion of solar radiation refracted light,the color of the fruit and the branches and leaves are similar,the branches and leaves are blocked,and the fruits are blocked and overlapped.For application requirements,a tomato target recognition and positioning algorithm based on binocular vision and deep learning is proposed.The main research content and results of this project are as follows:(1)Research on image processing technology based on mature tomato target recognition.Under different time,angle and distance conditions,the Bumblebee Xb3 binocular camera was used to obtain 10560 tomato images in the actual application environment.After the comparison and analysis of the three color space models of RGB,HSV and Lab,the collected image data was preprocessed,Use image enhancement and image filtering to improve the quality of the collected pictures,and finally use the Label Img software to calibrate the processed images to establish a training set and a test set.The results show that the difference between the tomato and the background of each channel of the RGB image is obvious,and rich texture features are retained;the tomato image is enhanced(histogram equalization)and median filtering to eliminate the noise in the original image.Improve the processing speed and computing power of the algorithm,and improve the recognition rate of the tomato target.(2)Tomato multi-target recognition research based on Faster-RCNN algorithm of VGG network.The small batch gradient descent method is used to train the YOLO V3 algorithm,the SSD algorithm and the Faster R-CNN algorithm.The results show that the YOLO V3 algorithm based on the Darknet-53 network has an AP value of 0.885 and a recognition speed of 0.107 s/frame in a steady state;based on the SSD algorithm of the VGG network,the model’s AP is 0.905 and the recognition speed is 0.096 s./Frame;Faster R-CNN is also based on the VGG network and uses the regional recommendation network(RPN)to detect targets.Its AP value is 0.930 and the recognition speed is 0.115 s/frame.Although the recognition speed is lower compared with the other two algorithms,However,the recognition accuracy is the highest,and there is still room for optimization in the future,so this paper finally chooses the Faster R-CNN algorithm with higher performance and high accuracy to apply to multi-target recognition in the actual environment.(3)Three-dimensional positioning experiment based on the center point of tomato picking.By studying the camera imaging model and the principle of dual target determination,using MATLAB to calibrate the binocular camera,the internal and external parameters of the binocular camera are obtained.Based on the principle of binocular stereo matching,the SIFT feature point matching method is selected to locate mature tomatoes,and The RANSAC algorithm is used to eliminate a large number of mismatched points and improve the accuracy of positioning.Finally,a three-dimensional tomato experiment is designed and implemented based on the principle of triangular ranging.The results show that the distance error between the actual measurement and the model measurement is within 15 mm,and the relative error is within 5%,which meets the needs of the automatic tomato picking robot vision system in the greenhouse environment.Based on binocular vision and deep learning,this paper has completed the identification and positioning of mature tomatoes,which lays the foundation for the research on the application of fruit and vegetable picking robots in the greenhouse environment.