Research On 3D Object Detection Algorithm Based On Fusion Of Point Cloud And Image

3d object detection is a key task in environment perception for autonomous driving,and has received much attention with the development of autonomous driving in recent years.Lidar point cloud and image are two main sensor data in this task.Since point cloud and image contain different types of scene information,detection based on fusion of them can lead to more accurate and reliable detection results using their complementarity.However,fusion of point cloud and image is challenging due to their difference in perspective and data form.3D object detection algorithm based on fusion of lidar point cloud and image is studied in this thesis.The main research contents and achievements are as follows:(1)There exists the problem of unmatched receptive field between point cloud features and image features when performing point cloud-image feature fusion in existing methods.Image features may contain too little foreground information or too much background information,making them less effective in complementing point cloud features,resulting in suboptimal effect of point cloud-image fusion.To solve this problem,two methods for point cloud-image feature are designed in this thesis,i.e.,feature pyramid attention fusion method and region attention fusion method.The feature pyramid attention fusion method builds an image feature pyramid,which outputs multi-scale feature maps with different receptive fields,and combines the corresponding multi-scale image features of the point cloud with the attention mechanism where the point features is the query item,in order to obtain the corresponding image features with matched receptive field for fusion.Region attention fusion method takes adjacent pixel area of the projected position on a single image feature map,and combines the image features in this adjacent area with the attention mechanism where the point features is the query item.When the receptive field of image features is inproper,the most effective image features of this scale is searched in the adjacent area for fusion.In addition,when the point cloud is represented by voxel,the region attention fusion method can also rectify the spatial correspondence between point cloud and image.(2)The designed point cloud-image feature fusion methods are applied to the object detection process,and a 3D object detection algorithm based on fusion of point cloud and image is proposed in this thesis.Evaluation on public datasets has demonstrated the effectiveness of the 3D object detection algorithm,and the improvement of the two point cloud-image feature fusion methods over the baseline fusion method.The two fusion methods are also compared and analyzed.(3)Deployment in C++ environment of 3D object detection algorithm in practical application is studied and practiced in this thesis.An open-source 3D object detection algorithm is taken as example,and deployed in C++ environment based on its Python source code to simulate the algorithm deployment in practical application.Experiments show that the inference speed of this algorithm is improved after deployment in C++environment.