Research On 3D Target Detection Algorithm Based On Image And Point Cloud Fusion

In recent years,with the continuous development of convolutional neural networks and three-dimensional data acquisition technology,three-dimensional target detection technology has become one of the core technologies in the field of computer vision and autonomous driving.Compared with the two-dimensional target detection technology,the pose estimation of the target in the three-dimensional space is more important for practical application scenarios.Compared with two-dimensional image data,three-dimensional point cloud data has the advantages of not being affected by occlusion or illumination changes and containing the pose information of the target,but it does not provide semantic information.Two-dimensional image data has rich semantic information,but it is easily affected by occlusion and illumination and does not contain the pose information of the target.Therefore,there is information complementarity between them.This paper focuses on the fusion strategy of image data and point cloud data and the three-dimensional target detection algorithm in-depth theoretical and technical research.The main work and contributions of this paper are as follows:(1)This paper studies target detection algorithms based on convolutional neural networks,including classic image-based two-dimensional target detection algorithms and point cloudbased three-dimensional target detection algorithms,and we compare them with experiments.(2)This paper studies and analyzes the existing image and point cloud data fusion strategy.Aiming at the problem that the existing methods mostly use global feature fusion and do not realize the deep fusion of image and point cloud features,we propose an image and point cloud feature fusion algorithm based on perspective projection and bilinear pooling interpolation(PBPI-Fusion).This algorithm not only makes full use of the geometric correspondence between image data and point cloud data,but also achieves a more finegrained local feature fusion between image and point cloud.It makes up for the problem of point cloud sparseness and reduces the probability of false detection of small targets.It can further improve the accuracy of 3D target detection based on multi-source data fusion methods.(3)This paper studies and analyzes the F-Point Net network and its limitations.The FPoint Net network uses image pre-detection of two-dimensional targets as the basis for point cloud three-dimensional target detection.It serially processes image data and point cloud data.Aiming at its shortcomings of insufficient comprehensive utilization of RGB image information and point cloud geometric information,we propose an improved threedimensional target detection network PBPI-Fusion Net.First,the image feature extraction network and the image point cloud feature fusion network are introduced on the basis of the F-Point Net network.It optimizes the point cloud instance segmentation network and the three-dimensional bounding box estimation network,and improves the angle regression performance of the network.Then,a 3D-2D bounding box projection loss function is proposed.The loss function uses the projection relationship between the 3D bounding box and the 2D bounding box of the detection target to further improve the accuracy of the parameter estimation of the three-dimensional bounding box.This greatly improves the performance of 3D target detection.