Research On Decision Level Fusion Of Lidar And Camera Based On 3D Target Detection

With the vigorous development of autonomous driving technology,environmental perception,as an important guarantee for the safe operation of autonomous driving vehicles,is increasingly concerned by the majority of autonomous driving vehicle’s researchers and producers.As a key technology at the level of environmental perception,target detection plays a more and more important role in the safe driving process of autonomous driving vehicles.Compared with two-dimensional target detection,three-dimensional target detection is more practical.As the most common sensors in autonomous driving vehicles,lidar and camera are often used in target detection tasks.Since both lidar and camera have some shortcomings in obtaining the detection target information,and the fusion of lidar and camera can effectively overcome the shortcomings,the research on the three-dimensional target detection method based on the decision fusion of lidar point cloud and camera image has great practical value.Among the existing methods based on lidar and camera fusion,data level fusion and feature level fusion are most likely to use multimodal information.Therefore,they are very sensitive to data unified alignment,which usually involves complex architecture,difficult implementation,large amount of calculation and difficult to meet the requirements of real-time.On the contrary,the construction of fusion system at the decision level is much simpler,because they include trained single-mode detection methods,which do not need to be changed,and only need to be correlated at the detection result level.This thesis mainly carries out the research on 3D target detection method based on lidar point cloud and camera image fusion from the decision level.The main research contents and innovations are as follows:Firstly,the fusion of lidar and camera is studied in this thesis.This thesis analyzes the common fusion methods:data level fusion and feature level fusion,puts forward the problems of these methods,and puts forward the fusion of lidar and camera at the decision level.Secondly,by analyzing the detection results of target detection using lidar and camera alone,a judgment index to realize the correlation between the detection results of the two sensors is proposed.In order to improve the final detection accuracy,the fusion of the results before non maximum suppression is proposed.According to the problems of traditional non maximum suppression methods in target candidate box screening,a new target candidate box screening method is proposed.According to the real-time requirements of target detection in practical application,a simple and fast fusion method is proposed,which is a fusion target detection method based on joint intersection over union and detection confidence score.Experiments show that the fusion method can effectively improve the accuracy of 3D target detection when the vehicle driving scene is relatively simple.Finally,a fusion target detection method based on attention convolution neural network is proposed to solve the problem of low accuracy of 3D target detection when there are many occlusions in the vehicle driving scene.The method of constructing sparse tensor is proposed to improve the speed of fusion processing;A method of increasing attention mechanism is proposed to improve the detection performance of convolutional neural network.Experimental results show that this method can effectively improve the accuracy of 3D target detection results.