| The automation of agricultural machinery requires that the operating machinery can perceive the surrounding environment and make autonomous responses based on environmental information.As an important environmental perception system for intelligent agricultural machinery,the machine vision system mainly used to identify and locate obstacles.Because the machine vision system has the advantages of wide detection range,high precision and high cost performance,the research on the vision system has now become a research hotspot in the field of agricultural machinery automation.Improving the detection accuracy and speed of the machine vision system is of great significance for the automation of agricultural machinery.The operating environment of mountain orchards is complex,and there are many small obstacles and trunks.Therefore,Ensuring the detection and positioning of small obstacles are the key to improve the engineering and intelligent level of intelligent agricultural equipment in mountainous areas.In view of the above problems,this paper carries out technological innovation to build a binocular visual obstacle detection system based on image semantic segmentation fusion network,which can accurately identify obstacles,especially small obstacles in the orchard environment at the pixel level.The integrated stereo vision system can reliably detect obstacles,and at the same time,the detection of the coordinates and distances of obstacles relatively accurate.This research provides basic support for the follow-up research on automatic navigation of intelligent equipment in mountain orchards.The main research contents and conclusions are as follows:(1)The construction of the visual system test bench and the establishment of the data set.The hardware system is composed of the mountain orchard operation chassis test platform,Lena Technology CAM-AR0135-3T16 binocular stereo vision camera,and high-performance PC;the semantic segmentation network and binocular stereo matching built under the framework of PyTorch and OpenCV are used Network Convergence Network construction software system.Before data collection,the orchard environment been investigated on the spot,and the specific obstacle detection requirements during agricultural machinery operation were analyzed.Based on these requirements,images been taken and processed to generate data sets.Because the data set collection can’t enumerate all orchard environment pictures,this study uses python programming to expand the data set,which expands the data volume by four times on the basis of the original data.The problem of over fitting in the training process of convolutional neural network been solved by data set expansion.After the expansion,the data set calibrated.The obstacles in the orchard scene divided into environmental background,pedestrian,tree trunk,stone,other small obstacles and other categories.Labelme software been used to annotate and establish the annotation data set.(2)Binocular camera calibration and image correction.The imaging model of the binocular vision system,the four camera coordinate systems and their mutual transformation relationships been analyzed,and the ranging principle of the binocular imaging system is analyzed to realize the dual-target determination to obtain the relevant parameters of the binocular camera.On the basis on comparing the OpenCV binocular calibration methods,the parameters obtained by Zhang Zhengyou’s calibration method was used as the parameters of the binocular camera in this study,and the reasons for the calibration errors are analyzed.The camera deformity correction been carried out by using the two-target fixed data,which solves the problems of distortion and unsatisfactory imaging effect caused by the camera’s own defects.After correction,the camera imaging effect reaches the expectation.(3)The semantic segmentation model of real-time image fusion built.Aiming at the problems of low efficiency of obstacle detection and false detection of small obstacles caused by complex working environment,a real-time fusion image semantic segmentation model proposed.Two independent branches extract the features of RGB image and depth image respectively.The RGB branch is the main branch and the depth branch is the auxiliary branch.The AFC module in this model used to fuse the information of different branches.Subsequently,through experimental verification,it is clear that the network model in this paper can run quickly and meet the real-time obstacle detection application requirements of orchard operation machinery.Finally,the experiment in the orchard operation scene shows that the proposed method is superior to the latest semantic segmentation model of the same type of image in terms of small obstacle detection.It has high accuracy and fast reasoning speed in high-resolution image recognition,and the obstacle detection performance of the proposed model is about 6.02% higher than that of the basic scheme.(4)Depth data reconstruction.By studying and comparing the advantages and disadvantages of traditional stereo matching algorithms with the latest existing stereo matching algorithms,BG-Net been selected as the stereo matching depth learning network in this paper.Combined with the real-time fusion image semantic segmentation model proposed in this paper,a semantic segmentation obstacle ranging system were built to realize the recognition,ranging and coordinate positioning of obstacles in the working environment.Finally,the experiment shows that the error of the system developed in this paper is stable within 3% within 2m.(5)System visualization and experimental analysis.The interface design of the orchard target ranging and recognition system carried out through the PyQt5 interface design tool.Through the obstacle detection and ranging experiments at different distances,the optimal detection range of the system is determined.The relative error between the measured value of the obstacle distance and coordinates and the real value measured at various distances to verify the accuracy of the system. |
PDF Full Text Request