3D Data Denoising Based On Deep Learning

With the widespread application of 3D scanning equipment and sensors,the media representation of the real world in the future will be more in the form of a combination of color and geometric information.3D will be the key way of data acquisition and presentation in various applications.In recent years,thanks to deep learning technology,the research on image analysis and generation has made great progress and has started to be commercialized.However,the research on 3D object analysis and generation is still”a long way off”,and relevant deep learning theories and methods need to be studied in depth and systematically.The typical sparse and irregular nature of 3D data exists.The sparsity is reflected by the fact that the geometric objects in real scenes usually need only a relatively small number of elements.The discrete and irregular nature of 3D data in the form of storage also poses more challenges for the application of deep learning techniques in the 3D domain.Therefore,how to make deep neural networks learn effective potential representations through geometric processing in sparse representations of 3D data is a key problem in 3D geometry learning.Geometric knowledge has been widely used in traditional 3D processing,and its effectiveness and robustness have been verified in several research works.Therefore,in this paper,from the perspective of geometric knowledge,deep learning techniques are used to investigate noise removal for two representations of 3D data: point clouds and triangular meshes.And we propose two denoising algorithms and verify the superiority of them on synthetic and real data.The two algorithms are described as follows.1.In this paper,we first propose a point cloud normal filtering algorithm based on robust height maps.Since the discrete irregular nature of 3D point cloud representation makes it difficult for deep learning techniques to learn robust geometric features directly on noisy point clouds,this algorithm seeks to combine robust geometric knowledge with deep learning techniques to achieve point cloud denoising.For each point in the irregular point cloud,the geometric neighborhood in its 3D space is projected to a specific plane,then each point in the neighborhood point set can get the corresponding projected height,and then an image interpolation algorithm is applied to generate a regular projected height map,which can be well combined with a convolutional neural network to perform normal regression.To improve the geometric feature-awaring ability of the algorithm,the normal tensor voting technique is used to construct a local coordinate system for the projection,so that the obtained height map can encode local geometric features more accurately,and then the point cloud normal filtering is performed through the residual network structure.Finally,the point coordinate update algorithm is used to adjust the coordinates of noise points to achieve point cloud denoising.For point clouds with different geometric features,this point cloud normal filtering algorithm can obtain better denoising effect.2.For triangle mesh denoising,this paper proposes a bi-domain denoising algorithm based on dual graph neural network.There is a dual graph structure in the triangle mesh,one of which is built based on vertices,i.e.,three-dimensional vertex as node in the graph structure,and the connected edges between vertices form the adjacency matrix of the graph structure;the other graph structure is built based on facets,i.e.,triangle facet as node in the graph structure,and a single facet and its neighbors in a certain range construct the connection relationship,thus forming the adjacency matrix.In this paper,by applying graph neural networks to these two graph structures separately,the vertex-based graph neural network learns the spatial domain information in the mesh representation and regresses the spatial coordinates of the vertices to achieve preliminary denoising;the facet-based graph neural network learns the normal domain information in the mesh representation and regresses the normals of facets,and then further updates the spatial coordinates of the vertices according to these normals to better maintain the geometric features.In addition,the method improves the graph pooling technique to further encourage the robustness to noise of the network.The algorithms proposed in this paper is validated on several publicly datasets containing multi-scale features and corrupted by noise at different scales.Both visualization and quantitative results show that the algorithm in this paper is a significant improvement over the state-of-the-art in terms of noise robustness and feature preservation.