Research On Key Technology Of 3D Reconstruction For Multimodal Indoor Scenes Based On Semantic Segmentation

Three-dimensional reconstruction has been a hot issue in the field of computer vision in recent years.Although many experts and scholars in this field are devoted to the research of 3D reconstruction,in the applications of augmented reality and intelligent interaction,the complex virtual environment brings hundreds of millions of Point Cloud data,and the complex point cloud There is a problem of mutual occlusion,which causes the existing 3D reconstruction method to process a large number of complex point cloud data,which will cause low reconstruction efficiency,low reconstruction accuracy,and poor reconstruction results.In indoor environments with many types of objects and more complex point clouds,such problems are particularly prominent.Therefore,in this paper,combining the relevant characteristics of 3D reconstruction in indoor environment,the traditional point cloud matching method is combined with a new type of semantic segmentation network,and a new 3D reconstruction method for indoor environment is proposed.The specific work is as follows:1.This article discusses the development status and research trends of 3D reconstruction research work firstly,and then combines the visual information collected by different visual sensors to discuss different types of 3D reconstruction.According to the characteristics of the current indoor environment 3D reconstruction,the depth sensor(Kinect)is used to obtain the depth data type of the indoor multi-view 3D scene,and the characteristics,defects and development direction of the current indoor scene 3D reconstruction are summarized.2.For the 3D data acquired by Kinect in the indoor environment,the reconstruction process requires high computing performance,low reconstruction accuracy,and high scene restriction.A three-dimensional reconstruction method of point cloud matching and point cloud sparse fusion is proposed.Based on dense point cloud matching,the multi-view color depth(RGB-D)image data is modeled,and the model is used to generate a three-dimensional point cloud of the indoor environment,calculate the reconstruction deviation and reconstruction efficiency,and then use the octree data structure to point Cloud space is compressed to construct a three-dimensional space Super Point model,and finally the scene image label is constructed by the super point model for three-dimensional reconstruction.3.This paper proposes a three-dimensional reconstruction method for indoor environment based on the fusion model of semantic segmentation and point cloud matching for various augmented reality 3D reconstruction applications,which have the disadvantages of poor real-time performance and poor reconstruction effect.Multimodal RGB-D images and corresponding multi-view image tags are used as training data,and semantic segmentation network(SegNet)is used for supervised learning.This method marks the spatial position information of objects under various perspectives,and finally uses the model to reconstruct the three-dimensional indoor environment.Theoretical analysis and experimental results show that the semantic segmentation and point cloud matching fusion model proposed in this paper can reconstruct a complex indoor environment in three dimensions,and can achieve a high reconstruction efficiency on the basis of ensuring good reconstruction results.Higher real-time performance and better reconstruction effect.