Camellia Oleifera Fruit Target Detection And Localization Based On Stereo Vision And Deep Learning

The existing picking methods of Camellia oleifera fruit are difficult to meet the production needs of the large-scale development of Camellia oleifera fruit industry.It is urgent to realize the mechanized and intelligent harvesting of Camellia oleifera fruit and improve the production efficiency.Rapid and accurate identification of the distribution position of Camellia oleifera fruit in the canopy plays a key role in the efficient and low-loss operation of Camellia oleifera fruit picking machine.However,the current fruit detection and positioning is still susceptible to light interference,which is difficult to meet the needs of rapid on-site positioning.Therefore in this thesis,the canopy Camellia oleifera fruit is taken as the research object.Based on binocular structured light vision,the visual positioning system of Camellia oleifera fruit is established.The positioning device of Camellia oleifera fruit based on embedded platform is designed and developed.The device is used to locate the target Camellia oleifera fruit in the image of canopy Camellia oleifera fruit in the field environment,combined with the visual positioning system of Camellia oleifera fruit,developed the detection software of embedded device.Finally,the field experiment was carried out to verify the function of the camellia fruit positioning device.The main research contents of this thesis are as follows:(1)The data set of canopy Camellia oleifera fruit was constructed and the recognition model of Camellia oleifera fruit was carried out.The images of Camellia oleifera fruit in complex environment of orchard were collected.The detection model of Camellia oleifera fruit was established by using YOLO v7,YOLO v5,YOLOv3-spp and Faster R-CNN target detection network.The best target detection network was YOLO v7,the m AP of the model was 95.74%,the F1 score was 93.67%,Precision was 94.21%,the recall was 93.13%,and the average detection speed was 0.025 s.Furthermore,the multivariate data enhancement method was used to expand the data set,and the generalization ability of the model was optimized to establish the DA-YOLOv7 detection model.The m AP was 96.03%,F1 score was 95.15%,Precision was 94.76%,and the recall was 95.54%.The correct recognition number of Camellia oleifera fruit under different illumination and occlusion conditions was the most,which could effectively avoid wrong and missed detection.(2)Determine the overall scheme and overall architecture of the camellia fruit visual positioning system based on the camellia fruit recognition model.The binocular structured light stereo vision system is constructed by using the depth camera OAK-D-Pro combined with camera calibration and stereo matching.The camera was calibrated by Zhang Zhengyou’s calibration algorithm to obtain the internal and external parameters,and the coordinate system conversion is performed on the obtained positioning points.Combined with the output detection frame of DA-YOLOv7 detection model,the centroid stereo matching of binocular structured light fruit image is completed.After stereo matching,the three-dimensional space point positioning test of Camellia oleifera fruit was carried out,and multiple sets of experiments were carried out to obtain depth coordinates and depth errors.The experiment shows that the average error value of stereo vision is 20.9 mm and the average relative error is 1.82% in the set range,which can meet the requirements of recognition and positioning of Camellia oleifera fruit.(3)Camellia fruit detection and positioning device based on embedded technology.According to the detection requirements in the field environment,the hardware and software of the embedded positioning device of Camellia oleifera fruit were selected to complete the connection of various components of the hardware.The Ubuntu operating system was deployed on the edge computing module,and the program design and graphical interface development of the positioning device are completed by using the Qt Creator development tool.(4)The field canopy camellia fruit identification and positioning test was carried out based on the embedded camellia fruit positioning device.The stability verification test of the developed camellia fruit detection and positioning device was carried out,and the camellia fruit detection and positioning test in the field environment was designed and carried out.The positioning coordinates of Camellia oleifera fruit were counted.The average recognition accuracy of the device under different recognition distances was 88.39 %.The average error of positioning in the field environment was 30.33 mm,and the average relative error was2.26 %.