Research On Key Technology Of Autonomous Driving Vehicle Perception On Road Environment

In order to navigate correctly and accomplish specific driving tasks,autonomous driving vehicles need to continuously perceive and understand the road environment based on a variety of sensors,and provide environment information for navigation safety.Road environment understanding,as one of the most important parts of the research on autonomous driving vehicles,is a necessary condition to ensure the safe and fast driving of autonomous driving vehicles.At present,although the related technologies are devel-oping,there are still many problems that can not be overcome.For example:1)In the current research on road segmentation,the segmentation method using a single sensor data is vulnerable to different weather,illumination and other environmental factors.The methods of extracting features manually are hard to be applied to different environments.Fully supervised learning methods need a large number of manual labeled training data.2)In most of the pedestrian detection and recognition methods,empirical parameters need to be set manually in the process of extracting the regions of interest(ROI),there-fore the parameters need to be adjusted manually for different data.3)There are still many challenges in semantic segmentation of road environment.For example,due to the complexity reflection textures on the water surfaces on the road,there is still not a good water segmentation method.Therefore,how to solve the above problems and improve the results of road environment understanding has become the key of related research.In this paper,aiming at the above road environment understanding problem of the autonomous driving vehicles,several key technologies including road area segmentation,pedestrian detection and recognition,water area segmentation,are studied via methods based on multi-sensor fusion,deep learning and generative adversarial networks.The specific research contents and innovations of this paper are as follows:1)Aiming at the problem that the road segmentation method using a singlesensor is susceptible to environmental noise,a road segmentation method based on Lidar and image data fusion is proposed.In this method,the Lidar point clouds are projected onto the image to generate sparse height maps,and then the dense height images are generated using bilateral filtering algorithm so that the data can be fused in the data layer.Subsequently,Textons features,location features and local distance distribution features are extracted,and the fusion in the feature level is completed by using conditional random fields to obtain the final road segmentation results.Experiments on KITTI road dataset show that this method has better performance than other methods.2)Aiming at the problem that the road segmentation methods based on tradition-al machine learning methods need to extract handcrafted features,a road segmentation method based on fusion of multi prior information and full convolutional neural network is proposed.This method is based on Resnet 101 deep neural network architecture.It can automatically extract features from color images and fuse a variety of prior information including location image,height image and gradient image to get the road segmenta-tion results.This method has been tested in KITTI road dataset and achieved good performance.3)A semi-supervised learning road segmentation method and a weakly supervised learning road segmentation method based on generative adversarial networks are pro-posed to solve the problem that the fully supervised learning road segmentation methods need a lot of manually labeled training data.The semi-supervised learning process and weakly supervised learning process are accomplished by iterativly optimize the genera-tors(road segmentation network and road shape prediction network)and discriminators(discrimination network)in the generative adversarial networks.In this method,a large number of unannotated images can be directly used to train the road segmentation net-work with only a small number of artificially annotated images.The experimental results on KITTI road dataset show that the proposed method can significantly improve the over-fitting problem,speed up the convergence rate,and reduce the work of manual labeling training samples,as well as achieve the best performance on this dataset.4)Aiming at the problem that to extract ROI in current pedestrian detection and recognition methods requires manual setting of parameters,a pedestrian detection method based on the fusion of Lidar and image data is proposed.This method uses Lidar data to automatically segment images and extract regions of interest.After that,the background texture is removed in all regions of interest.Then the histogram of ori-ented gradient feature based on the images and the geometric feature based on lidar point clouds are extracted,and the SVM classifier is trained to detect pedestrians using the fused features.This method is tested on NJUSTRobot dataset and KITTI pedestrian dataset,and the effectiveness of the proposed method is verified.5)Aiming at the problem that the water areas are difficult to segment because of the changeable reflection textures on their surfaces,this paper proposes a single image water area segmentation method based on reflection attention unit.The reflection attention unit proposed in this method can help the deep neural network to learn the reflective re-lationships between different parts in the vertical direction,and improve the performance of water area segmentation.At the same time,985 images with different sizes and shapes of water areas are annoataed and compose the Puddle-1000 dataset.This method is the first water area segmentation method based on deep neural network.Experiments on this dataset show that the proposed method has excellent performance.