Data-driven Indoor Scene 3D Reconstruction And Semantic Understanding

Indoor scene reconstruction and understanding has been a difficult problem in computer vision,computer graphics and robotics community.Indoor scene understanding algorithms are the core of several tasks,such as robot navigation and knowledge reasoning.It is a fundamental component of indoor service robot.As the growing demand of digital geometry techniques in industry and entertainment communities,indoor scene understanding is more important than ever.The recent development of depth scanning has greatly widen the way of data acquisition for indoor scenes.This gives the birth of the recent trend of complex and high-level indoor scene reconstruction and understanding.Indoor scene reconstruction and understanding via 3D data is a hot and difficult issue at the current stage.On the other hand,with the emergence of big data,data-driven approach has played an important role in a significant number of areas.It has been proved that data-driven approach is an effective tool for high-level semantic analysis of geometry data.Due to the increasing number of 3D data,more and more data-driven approaches has been proposed to address 3D indoor scene reconstruction and understanding problems.In this paper,we study data-driven indoor scene reconstruction and semantic understanding approaches.We first study the indoor scene reconstruction by coplanarity prediction of planar patches.We then study the indoor scene semantic understanding by object detection via recursive autoencoder.We also study the proactive object analysis by developing high quality single object reconstruction.The main contributions of this thesis include:1.Patch coplanarity prediction network for 3D scene reconstruction: Patchbased constraints could provide additional signal and constraints which are critical in many scanning scenarios when the key-point fail to find loop closures.Improving the quality of 3D reconstruction by adding patch coplanarity constraints is an important solution of this problem.We propose a deep network to produce features for predicting whether two image patches are coplanar or not,and a robust optimization algorithm to solve camera poses with coplanarity constraints.The proposed method outperforms existing methods,in terms of reconstruction quality,by a large margin.2.3D scene semantic understanding via recursive autoencoder: Structural and contextual information plays an important role in indoor scene analysis.We propose an3 D scene semantic understanding method based on recursive autoencoder.This method first organizes the input scene by a hierarchy.A recursive autoencoder is applied to the hierarchy to aggregate and propagate contextual information,thus detecting objects in the scene.Experiments show that the proposed method could aggregate contextual and structural information.Moreover,it wins the first place on two 3D object detection datasets.3.Proactive scene reconstruction and analysis by robot:We propose a proactive autoscanning approach by using a robot.In our method,scene reconstruction is coupled with object analysis: the scene reconstruction provides data for object analysis,and the object analysis helps to evaluate the reconstruction quality.The robot first scans for a rough reconstruction of the entire space,and then compute the reconstruction and segmentation confidences.For the areas with low confidences,the robot performs a horizontal push to verify the object analysis result.The object analysis is built on an online learning model,whose parameters will be updated by the new observations acquired by the robot in an online manner.These processes iterates until all the objects has high confidences both on reconstruction and segmentation.