Research On Multi-scale Geometric Deep Learning Method Based On Point Cloud And Voxel Fusion

With the development and popularization of depth sensing devices,the 3D point cloud data generated by them are widely used in the field of computer graphics,such as augmented reality/virtual reality(AR/VR),robotics,and autonomous driving.Over the past few decades,perception of 3D point cloud data has highly relied on manual analysis methods,such as designing hand-crafted features.However,with the vigorous development of deep learning,more and more researchers have introduced deep learning into the analysis of point clouds.Recently,researchers have employed deep learning models to handle such tasks,significantly improving the perceptual performance of current 3D point cloud data.At present,in complex application scenarios,the research on timely execution of 3D point cloud perception models with low computing resources has attracted more and more attention.But the point cloud has disorder,irregularity and sparseness.The conventional method is to convert the irregular point cloud into a regular voxel grid,but the complexity of the voxel grid will appear with the increase of voxel resolution.exponential growth.This paper improves and optimizes the traditional 3D convolution block,and designs a lightweight 3D convolution block Rep PVConv.The main work is as follows:(1)In view of the disorder and irregularity of point cloud,a lightweight 3D convolution block Rep PVConv is designed,which is composed of two branches based on point cloud and based on voxel,and the branch based on voxel Neighborhood features are extracted by transforming irregular and disordered point clouds into regular and ordered low-resolution voxel grids;the point cloud-based branch extracts high-resolution features of individual raw points to complement the volume prime features.The 3D convolution block Rep PVConv can effectively improve the perceptual performance of 3D point cloud data.(2)Aiming at the problem that the high-resolution voxel grid is too complex and the voxel feature extraction is insufficient at low resolution,two reparameterizable 3D convolution modules are designed.When training the model,multiple Convolution kernels of different sizes are used to fully extract voxel features under multiple receptive fields.During inference,multiple convolution kernels of different sizes are fused into one convolution kernel to reduce the complexity of the model,thereby greatly improving the efficiency of 3D network inference.(3)In order to effectively fuse the voxel-based branch and the point cloud-based branch,an attention module is proposed.The traditional fusion method uses the trilinear interpolation algorithm to return the voxel features to the point cloud.A linear method for calculating weights by voxel grid-to-point cloud distance,which causes voxel features that are too far away but important to be ignored.The attention mechanism is able to enforce efficient fusion of 3D point cloud and voxel features in a non-linear fashion,resulting in higher feature extraction accuracy.