Research On Indoor Scene Modeling From A Single Image

As one of the fundamental problems in computer graphics and computer vision,indoor scene modeling has been extensively studied,due to its wide applications to smart home,interior design,VR/AR,indoor robot,and digital entertainment,etc.The core to indoor scene modeling is to reconstruct the 3D information of the indoor scene with visual mediums including image,video and depth as input.Compared with im-age sequences,videos,and RGB-D data,a single image conveys less even ambiguous scene information,making single-image-based indoor scene modeling more difficult and challenging.The key for single-image-based indoor scene modeling is to extract effective features from the input image and reconstruct the 3D geometry with the ex-tracted image features.In this thesis,with a single indoor image as input,we use a data-driven manner by the aid of 3D model database.Starting with investigating the relationships between the objects in the input image and 3D models from the database,we extract effective image features and further retrieve the similar 3D models.The spatial locations and poses of the retrieved 3D models are then optimized,generating the final modeling results.More specifically,our work mainly contains the following aspects:1.A novel method for indoor scene modeling using normal inference and edge features is proposed.Our key insight is that,although depth estimation from a single image is notoriously difficult,we can conveniently obtain a relatively accu-rate normal map,which essentially conveys a great deal of scene geometry.This en-ables us to model each object in a data-driven manner by representing the object as a normal-based graph and retrieving a similar model from the database by graph match-ing.Moreover,edge information is integrated to further improve the searching result.With a small amount of simple user interaction,our approach is able to generate a plau-sible model of the scene.To verify the effectiveness of our proposed method,we show the modeling results on a variety of indoor images.2.A novel method for image-based 3D model retrieval by simulating scene context is proposed.For the input indoor image,the proposed approach retrieves the most similar 3D models from the ShapeNet model repository,and aligns them with the corresponding objects automatically.By simulating the scene context of the input image,our method is able to handle several challenging scenarios featuring cluttered backgrounds and severe occlusions.Concretely,each 3D models is first represented by calibrated view-dependent visual elements learned from the rendered views.With the estimated occlusion relationships,the rendered model images are then assembled to form a new composed scene to simulate the scene context.By conducting matching between these composed scenes and the input image,the most similar 3D models un-der the approximate poses are finally retrieved.Experimental results on public datasets demonstrate the effectiveness of our proposed method.Moreover,we show that the retrieving time can be significantly reduced based on a novel greedy algorithm.Exten-sive experiments on synthesized images show that the proposed algorithm achieves a high retrieval accuracy,outperforming state-of-the-art methods.3.A novel method for indoor scene modeling with iterative object segmen-tation and model retrieval is proposed.We propose a new method for modeling the indoor scene from a single color image.With our system,the user only needs to drag a few semantic bounding boxes surrounding the objects of interest.Our system then automatically finds the most similar 3D models and aligns them with the corresponding objects of interest.We iteratively conduct object segmentation and 3D model retrieval,based on the observation that good segmentation of the objects of interest can signif-icantly improve the accuracy of model retrieval and make it robust to cluttered back-ground and occlusions,and in turn,the retrieved 3D models can be used to assist with object segmentation.Segmentation of all objects of interest is achieved simultaneously under a unified multi-labeling framework which fully utilizes the correspondences be-tween the objects of interest and retrieved model images.Besides,we propose a new method to estimate the scene layout of the input image with the segmentation masks,which helps compose the resulting scene and further improves the modeling result re-markably.We verify the effectiveness of our approach through experimenting with a variety of indoor images and comparing against the relevant methods.