Research On Semantic Understanding Of Indoor Point Cloud Scenes Based On Deep Learning

Semantic understanding of indoor scenes is important for the development of intelligent devices such as robots.But the difficulty of semantic understanding is increased,with indoor scenes containing a large number of complex structures and a variety of artifacts.The traditional methods of semantic understanding based on 2D images are strongly influenced by camera parameters or uncontrolled illumination,and the level of semantic understanding is low.Solving the difficulty of indoor scenes semantic understanding,getting the semantic categories in scenes and instances in each category will make a new breakthrough in the field of computer vision,e.g.,navigation,man-machine interaction,environmental perception and 3D digital modeling.With the convenience of depth data,solving the problem of indoor scenes semantic understanding is mainly discussed in this paper.With the help of deep learning and the combination of 2D color information and 3D depth information,the semantic understanding framework of indoor scenes is proposed,including 2D semantic segmentation,semantic scene reconstruction and 3D instance segmentation.In order to obtain the semantic categories and combine the advantages of RGB and Depth,a 2D double-data-flows semantic segmentation network based on U-Net is designed in this paper.Through analyzing evaluation indexes,comparing different data types and network models,the network is verified with effectiveness in indoor scenes semantic segmentation.In order to provide a complete 3D semantic data for instance segmentation,reduce the amount of data and speed up data processing,A 3P-ICP splicing algorithm for small amount of discrete semantic point cloud frames is proposed,which combining the advantages of Three Points method and Iterative Closest Points algorithm,this algorithm ensures the accuracy of splicing process,and carries out the semantic scene reconstruction from 2D to 3D.Considering that different instances of same category often have spatial location gaps in 3d semantic scenes,An improved threshold clustering segmentation algorithm based on Euclidean distance is proposed to obtain the instances,This algorithm can accurately segment the real semantic point cloud data with noise and semantic errors,and finally complete the construction of semantic understanding framework.In this paper,the semantic understanding framework is applied to automatic combination CAD modeling of indoor scenes.Through model analysis,various existing CAD object models are matched and combined,which realizing the automatic and highfidelity CAD modeling of indoor scenes,and verifying the validity of semantic understanding framework.