Front Vehicle Recognition Method Based On Lidar And Image Heterogeneous Data Fusion

In the process of intelligent vehicle driving,fast and accurate detection of dangerous targets in the surrounding traffic is the basis for intelligent driving,and it is also one of the key problems that the current intelligent vehicle technology needs to solve.In this thesis,a new method of front vehicle detection based on multi-sensor fusion is proposed to solve the problems that single sensor can not get enough data and the current multi-sensor fusion algorithm can not meet the application requirements.The main works are as follows:(1)The convolution neural network is analyzed theoretically,and the selection and processing of data set are introduced.In order to better understand the basic principle of depth completion and target detection algorithm based on deep learning,this thesis analyzes the convolution neural network theory,and expounds the basic structure of convolution neural network in detail.(2)A deep completion model based on FusionNet is built.In this model,the network structure of FusionNet is optimized in the form of grouping convolution and expanding convolution.RGB image and sparse point cloud image are used as inputs,codec is used as the main structure of the network,and dense point cloud image is obtained as output.Compared with the similar depth completion model,the model performs well in accuracy and real-time performance.In order to facilitate the follow-up research,the data set used in this thesis is selected and processed.KITTI depth completion and target detection data set are selected in this thesis,and the image data,label data and point cloud data in the data set are preprocessed.(3)An image fusion algorithm based on Wavelet Transform is built.In this thesis,in order to increase the feature information in the image,RGB image and HSV image are fused.The low-frequency and high-frequency coefficients of the image are fused by weighted average method and local variance method respectively.The wavelet basis function and wavelet decomposition level are selected based on the objective evaluation parameters of the fused image.Compared with the original RGB image,the fused image contains more information and reduces the influence of illumination.(4)A vehicle target detection model is constructed.In this thesis,the lightweight model is taken as the optimization direction,Mobile Net V2 network instead of Darknet53 is selected as the backbone network of YOLOv3.The multi-scale fusion structure is improved,which connects the dense point cloud image and the fusion image as the input of the target detection model.Compared with the YOLOv3 model,the detection performance of the optimized model is greatly improved.At the same time,compared with the single RGB image input and single fusion image input,the HSV feature and depth feature integrated into the input image can significantly improve the vehicle recognition accuracy.