Research For Field Road Scene Recognition And Obstacle Detection In Hilly Areas Based On Vision

Machine vision system is one of the main environmental perception equipment carried by intelligent agricultural machinery.Its main function is to detect drivable areas,obstacles or crops,so as to realize automatic navigation and obstacle avoidance of intelligent agricultural machinery.In hilly areas,the road widths are varied,the road morphological changes are complex,the road curvature is large,the road surface is undulating and bumpy,and obstacles such as weeds and soil along the road are scattered,etc.,all of which bring great difficulties to navigation and obstacle avoidance.Aiming at the characteristics of unstructured roads in the field,such as twists and turns,no lane lines and obvious boundaries,this paper proposes a road and obstacle recognition method based on the combination of dilated convolutional neural networks and binocular stereo vision,which provides a practical basis for the navigation and obstacle avoidance of intelligent agricultural machinery based on machine vision in hilly areas.The main contents and conclusions of this paper are as follows:(1)Construction of visual system platform.We take the automatic moving field road carrier developed in the early stage as the experimental platform.The RER-1MP2CAM002 parallel binocular stereo vision camera produced by Rervision Company was added to the platform,high-performance PC was used as the image processing system and various hardware and software equipment were integrated to build the semantic segmentation and obstacle detection platform in this paper.Based on the analysis of characteristics of field road images in hilly areas,we divided field road scene objects into 11 categories: background,road,pedestrian,vegetation,sky,building,cattle,obstacle,pond,soil and pole,and collected images to build data set.In view of the over-fitting situation that was easy to occur in CNN training,we used the operation of data augmentation to increase the amount of data.(2)Construction of dilated convolutional neural networks.The field road scene in hilly areas is complex,with many kinds of targets in the scene,the road edges are mostly covered by weeds and crop branches and leaves,and the shadows on the road surface vary greatly.The classical FCN(fully convolutional networks)are adapted from the traditional classification network,and its semantic segmentation effect can not meet the requirements of field road pixel-level segmentation.This paper proposes to use DCNN(dilated convolutional neural networks)for image segmentation of field scenes in hilly areas.Based on the VGG-16 structure in FCN,we proposed a front-end module with higher prediction accuracy by removing the parts unfavorable to pixel prediction in the classification network and modifying some convolutional layers,and designed two kinds of context aggregation modules based on dilated convolution to combine with the front-end module.(3)Field road scene recognition experiment in hilly areas.In CAFFE,a deep learning framework,we built improved networks and a FCN-8s network based on VGG-16,and carried out comparative tests on FCN-8s,front-end,front-end + basic and front-end + large to test the effectiveness of the front-end module and the context module in improving the prediction accuracy.To improve the problems of long training time and slow convergence speed of deep learning network,two-stage training is adopted.The segmentation results show that the improved DCNN network structure has a good effect.The pixel accuracy of the front-end + large model can reach 88.5%,and the mean IoU can reach 74.2%,which are respectively 7.6% and 8.9% higher than the traditional FCN-8s network.Meanwhile,it has a good test effect on the field road shadow interference,indicating that the model has good generalization and robustness,and can realize the pixel-level prediction of field road images in hilly areas.(4)Obstacle detection based on binocular stereo vison.We selected the classic Zhang Zhengyou calibration method,and calibrated the RER-1MP binocular camera in the Camera Calibration tool box of MATLAB.Comparing with the pixel errors of 8 groups of calibration distance,the intrinsic and extrinsic parameters of the camera were calculated and optimized.After stereo rectification with Bouguet algorithm,we used the semi global block matching(SGBM)algorithm in OpenCV for binocular stereo matching.The obstacle detection method based on disparity map was used to eliminate the interference objects such as the areas outside the road,the shadows on the road and the passable obstacles,and the three-dimensional information of the obstacles is extracted.After obstacle detection tests at different distances and different obstacle detection tests,the optimal obstacle detection distance was determined.The actual measurement of the distance,width and height of obstacles showed that the average relative errors were-2.68%,-0.98% and-1.34%,respectively(5)Calculate the navigation line for the passable area.Extracted the road part and non-road part with the highest pixel accuracy in the semantic segmentation results,and conducted morphological filtering and connected domain processing to obtain more complete road area and smooth road boundary;According to the three-dimensional information of obstacle detection,the current road passability was judged;The centroid method was used to obtain the centroid points of the passable road area,and the navigation line was fitted by the least square method.On this basis,the accuracy of the navigation line under different road shapes and obstacles in the field in hilly areas was tested,and the relative error was within the range of 0.115-4.808%,which could meet the error requirements of the automatic driving navigation line of the field road carrier.The field road scene recognition model based on DCNN constructed in this paper can accurately identify the field road at the pixel level.The binocular stereo vision system has high reliability in obstacle detection,and the generated navigation path is precise,which provides basic support for the following research on the automatic navigation of field road carrier in hilly areas.