Research On Cross-modal Point Cloud Completion Based On Deep Learning

3D point cloud completion is a very important research field in computer vision.Traditional deep learning-based point cloud completion methods are limited to training on a single point cloud modality,which often results in poor completion performance in cases of large missing regions and severe occlusions.With the widespread use of mixed sensors,cross-modal point cloud completion has become very promising,as it can use different modalities of point cloud data to complete missing parts and improve result reliability and completeness.Cross-modal point cloud completion tasks require addressing two problems: how to fuse features from multiple modalities,known as the cross-modal problem,and how to match global and local missing features,known as the cross-level problem.Thesis proposes a novel deep learning-based cross-modal point cloud completion network model,which uses a dual-path structure to fuse views and incomplete point cloud features for point cloud completion.The model consists of four main modules: the view reconstruction module extracts view information and reconstructs a 3D coarse point cloud to provide regional detail features.Thesis developed module constructions for both singleview and dual-view to address the cross-modal problem.The region segmentation and feature extraction module perform semantic consistency segmentation and feature extraction on the point cloud reconstructed from views,for more targeted training of subsequent fine point cloud generators.The global feature extraction module uses DGCNN to extract features from incomplete point clouds.To better learn the shape’s detail information,thesis adopts the block-based point cloud generation method,where the global and regional features are fed into sixteen non-parameter sharing fine point cloud generator modules for regional prediction,which is then merged to form the completion result,addressing the cross-level problem.In order to obtain more fine-grained results,Thesis also constructs a cross-modal point cloud completion specialized loss function based on projection to limit the occurrence of outliers in the completion results.Thesis conducted extensive ablation experiments,numerical quantification,and visualization experiments on the eight main categories of the ShapeNet-ViPC dataset to verify the stability of the overall model and the effectiveness of each module.Comparative experiments also demonstrate that the proposed network model outperforms previous methods in both reconstruction loss and F1 score and is the first study in the cross-modal point cloud completion field to use multi-view image information for completion.