| Architectural visualizations of indoor scenes enable compelling applications in several areas, such as real estate, interior design, cultural heritage preservation, and more recently, immersive virtual reality. Computer-Aided Design (CAD) tools have been invaluable for creating such visualizations. However, creating detailed, attractive visualizations of scenes remains a challenging task, particularly for non-experts. User interfaces for CAD tools tend to be complex and require significant manual effort to operate. These tools are also designed to be used ex-situ, or off-site, making it difficult to record and reproduce details faithfully.;In this thesis, I propose novel techniques and systems for interactive in-situ scene capture on mobile devices that let non-expert users quickly and easily capture useful architectural visualizations of indoor scenes. These systems are built upon two key insights; 1) sensors on mobile devices can be leveraged to capture important aspects of the scene such as dimensions, room shape, furniture placement, etc., and 2) an in-situ user can assist in the modeling task by acting as a guide for reconstruction and object recognition algorithms. Based on these insights, the semi-automatic systems that I propose combine the strengths of the user, who is good at high-level semantic reasoning, and the computer, which excels at combinatorics and numerical optimization.;I present three systems in this thesis. First, I present a smartphone application designed to visually capture homes, offices and other indoor scenes. The application leverages data from smartphone sensors such as the camera, accelerometer, gyroscope and magnetometer to help reconstruct the indoor scene. The output of the system is two-fold; first, an interactive visual tour of the scene is generated in real time that allows the user to explore each room and transition between connected rooms. Second, by marking distinct room features such as corners and doors, the system generates a 2D floor plan and accompanying 3D model of the scene, under a Manhattan-world assumption. This approach does not require any specialized equipment or training, and is able to produce accurate floor plans.;I then describe an interactive system to capture CAD-like 3D models of indoor scenes, on a tablet device. The modeling proceeds in two stages: (1) The user captures the 3D shape and dimensions of the room. (2) The user then uses voice commands and an augmented reality sketching interface to insert objects of interest, such as furniture, artwork, doors and windows. The system recognizes the sketches and add a corresponding 3D model into the scene at the appropriate location. The key contributions of this work are the design of a multi-modal user interface to effectively capture the user's semantic understanding of the scene, a framework for sketch based model retrieval, and the underlying algorithms that process the input to produce useful reconstructions.;Finally, I extend the in-situ modeling approach to 3D-aware mobile devices, an emerging class of devices that can more richly sense the 3D nature of our world using depth sensing and self-localization. I propose a novel interactive system to further simplify the process of indoor 3D CAD room modeling on such devices. The proposed system leverages the sensing capabilities of a 3D aware mobile device, recent advances in object recognition, and a novel augmented reality user interface, to author indoor 3D room models in-situ. With a few taps, a user can mark the surface of an object, take a photo, and the system automatically retrieves and places a matching 3D model into the scene, from a large online database -- a modality that proves to be faster, more accurate, and easier than using traditional desktop tools. |
