Semantic Based 3D Reconstruction Of Indoor Scenes

With the widespread application of ubiquitous location-based services(LBS),the low-cost and efficient large-scale indoor 3D model reconstruction has become essential tasks.However,compared to the outdoor reconstruction system that can efficiently output a city-scale model from one sampling,such as the long-range photographs taken by unmanned aerial vehicles(UAVs)or the street images captured by moving survey vehicles,the indoor surveying methods can only obtain a short-range model in a limited space,which intrinscically limits the reconstruction efficiency of the indoor models.Therefore,the technologies that can automatically construct large-scale geometrical-consistent indoor 3D models has become impervasive.Reccent indoor 3D model reconstruction methods rely on light detection and ranging(LiDAR)surveying,Kinect collection to produce high quality 3D indoor models.However,these methods require special instruments and professional operation and therefore could not serve as an effective solution for the vast requirements of indoor reconstruction.In order to achieve ubiquitous indoor reconstruction,the low-cost and effective image-based 3D model reconstruction technology has received extensive attention.Due to the indoor complexity,the traditional image-based model reconstruction methods still face some challenges:(1)Low efficiency in model reconstruction.The traditional method uses all the images to construct the model at one time,which involves redundant pair-wise image matching and exponentially increase the computation complexity.(2)Difficulty in semantic extraction.Due to the complexity of indoor structures and the difficulty in constructing 3D feature descriptors,the semantic extraction and object-oriented classification of 3D point cloud models become difficult,resulting in "blind" point clouds with precise geometry but unknown semantics.(3)Deficiency in model geometry.Due to the low texture,high occlusion,and changing illumination(high light,reflective light)problem of indoor images,the point cloud model contain holes and noises,making the purely geometric-based surface reconstruction method unable to obtain a complete indoor surface model.Therefore,an effective methods that can produce dense and complete indoor models is highly demanded.Targeting the problems of indoor 3D model reconstruction,an end-to-end indoor model reconstruction method that output surface model from images is proposed in this dissertation.The main contents and innovations include:(1)The image-based 3D reconstruction methods are systematically summarized.The dissertation analyzes the related methods and the problems of indoor 3D model reconstruction,and detaily discusses the image-based indoor 3D reconstruction methods.(2)An annotated hierarchical SfM algorithm is proposed for 3D reconstruction of indoor scenes.Because the complexity of indoor space,as well as the computational complexity of traditional incremental SfM algorithm,this dissertation uses bag of visual word(BOVW)model and support vector machine(SVM)to classify the images into small and tracable subsets,and reconstructs each 3D model independently.Then the RANSAC generalized Procrustes analysis(GPA)is used to merge the separate models into a complete structure.This method avoids the redundant pairwise image matching in traditional SfM and can improve the model reconstruction efficiency.(3)A semantic propagation from 2D image to 3D point cloud is proposed to label 3D point clouds.While the 3D point cloud model labeling is often compromised by the difficulty of desiging effective 3D feature descripors and sufficient training data,the dissertation instead makes use of recent complishments in 2D image classification and propagates the semantic information to 3D point cloud via a graph model.Based on the semantic classification and spatial layout of convolutional neural network(FCN),a markov random field(MRF)graph model encoding both intra-image consistency and inter-image consistency is utilized to propagate the semantic information from 2D image to 3D model.(4)A surface model reconstruction algorithm that incorporates semantic and geometric priors is proposed.Traditional surface reoncstruction algorithms are often impossible to recover high-quality surface model from noisy and weakly reconstructed point cloud.However,the semantic information and class-specific priors can help to achieve robust surface reconstruction,by imposing reasonable regularizers.By incorporating the class-specific priors into the reconstruction algorithm,the dissertation can obtain the 3D surface model with high geometric accuracy and completeness.(5)A 3D model based visual localization approach is demonstrated.To investigate the applicability of 3D models in location-based services,an indoor localization system that fuses image-based localization and pedestrian dead reckoning(PDR)is presented.The experiments demonstrate that with the 3D point cloud database,it is possible to provide indoor localization services on smart phones,without the need of any dedicated infrastructure.To demonstrate the performance of the proposed approaches,the dissertation uses smartphones and SLR cameras to collect images as experimental data and reconstructs indoor 3D model.Compared with the classical SfM algorithm,the experiment results show that the proposed annotated hierarchical SfM algorithm can greatly improve the modeling efficiency,and therefore could serve as a low-cost indoor 3D model reconstruction strategy.In terms of semantic labelling of 3D point cloud models,the proposed approach can achieve accurate indoor point cloud labelling without requiring 3D training datasets,which provides new ideas for obtaining rich indoor semantic information.Moverover,compared with other classical 3D surface reconstruction methods,the proposed approach exploits semantic and geometry priors,and can better compensate the holes in the point cloud model and obtain a complete surface models.In conclusion,these algorithms provide an important data foundation for the development of indoor GIS systems and indoor localization services.