Road Obstacle Detection Of Intelligent Vehicle Based On Multi-sensor Fusion

In recent years,driverless technology has become the focus of domestic and foreign technology companies and the automotive industry.Its core technologies include environment perception,precise positioning,path planning,decision-making control,involving sensor,automatic control,communication and other technologies.Obstacle detection and classification is an important part of the environment perception of autonomous vehicle,which plays an important role in the safe driving of autonomous vehicle.But the complex road environment,especially the mixed traffic environment,makes it very difficult for the environment perception of autonomous vehicle.The disadvantages of single sensor scheme are low sensing accuracy and poor robustness,which make it difficult to meet the needs of unmanned driving for environmental perception.The existing multi-sensor fusion schemes are difficult to balance the detection accuracy and real-time,and it is difficult to put into engineering use.This paper based on the intelligent driving vehicle platform,and an environment perception scheme based on the fusion of lidar and camera is designed,which takes into account the detection accuracy and real-time performance,and provides reliable road target information for autonomous vehicles,so as to ensure driving safety.The main research contents are as follows:(1)Multi sensors calibration.Through the camera calibration,the internal parameter matrix is obtained;By solving the corresponding relationship between multiple pairs of points and image pixels,the rotation matrix and translation matrix between the lidar coordinate system and the camera coordinate system are obtained,and the external parameter calibration between multiple sensors is realized,that is,space synchronization.Time synchronization is realized based on lidar time stamp.The projection from point cloud to image shows that the calibration effect is good.(2)Fusion algorithm design.In view of the shortcomings of the existing multi-sensor fusion scheme of lidar and camera that is difficult to balance the detection accuracy and real-time performance or only a single kind of object can be detected,a fusion scheme of lidar and camera is designed: The lidar is used to detect and cluster the obstacles,the region of interest on the image is determined according to the location of the obstacles,and the improved single-stage neural network model is used to detect the objects in the region of interest.The target point cloud is obtained by matching the visual detection results and point cloud clustering results,and classified by neural network.Finally,the point cloud classification results and visual detection results are soft weighted average,and the final detection results are obtained.On the basis of ensuring the accuracy of the detection,it has good real-time performance and practical value of engineering.(3)Region of interest extraction.The algorithm of Max-min elevation map is used to extract the obstacle point cloud,then a grid clustering method based on dynamic distance threshold is proposed to cluster obstacles which effectively reduces the influence of lidar fixed horizontal angle resolution on clustering.After that,the obstacle point cloud is projected to the image and the corresponding region of interest of each obstacle in the image is enlarged based on the dynamic distance threshold,and the final region of interest is obtained after merging.(4)Visual target detection.An improved neural network model for target detection is proposed.In order to ensure the detection accuracy,the neural network model is selected as the detection model,and GIoU(Generalized Intersection over Union)and Soft-NMS(Soft Nonmaximum Suppression)are used to improve the positioning performance of the model and the detection performance of overlapping targets;In order to ensure the real-time performance,based on the single-stage model,the parameters in the batch normalization layer are used as the pruning factor to prune the model,and prune the convolution of cross layer connection with the strategy of union.Finally,finetune the pruned model.Experiments show that,on the basis of improving the detection accuracy,the volume of the model becomes 19.25% and the inference time becomes 36.10%,which greatly improves the performance of the model.(5)Point cloud classification and soft weighted average.Firstly,reduce the output confidence judgment threshold of visual detection results,so that some visual detection targets with lower confidence can also be output;By matching the clustering results of the obstacle point cloud and visual detection results,the obstacle point cloud corresponding to the visual detection target is extracted,which overcomes the interference of the point cloud generated by occlusion and other situations to the target point cloud;In order to ensure the real-time performance,a lightweight neural network model is designed;The problem of the disorder and rotation of the point cloud is overcome by using coordinate normalization,convolution kernel of specific size,maximum pooling;Finally,the results of visual detection and point cloud classification are soft weighted average,and the final detection results are obtained.Experiments show that the accuracy of this method is improved by 17.4%,20.9% and 2.5% respectively compared with the existing detection algorithms such as YOLOV3?Voxelnet?F-PointNET.At the same time,it ensures the real-time performance and has good engineering application value.