Sketching Reality: Realistic Interpretation From Freehand Sketching

Achieving realism is one of the major goals in computer graphics. Many approaches, ranging from physical-based modeling for light and geometry to image-based rendering, have been proposed. However, it is cumbersome and time-consuming to create new content for rendering when the scene is being designed. Sketching appeals as an aesthetic way because of its effectiveness in representing, exploring, and communicating concept ideas, especially at the first stage of design. Sketch understanding and modeling has been the one of the difficult and hot problems in computer vision and computer graphics for long time. A new concept sketching reality is been presented in this dissertation. The main idea is to convert the 2D freehand sketch to geometry model and render realistic imagery. More specifically, this dissertation proposed sketch-based modeling system for architectural design and tree modeling respectively. The details is described as following:1. Sketch-based realistic interpretation of architectural design. A general framework based on Maximum Likelihood (ML) is presented to interpret different types of objects in architectural drawing. The objects in an architectural drawing are divided into three types: primitive geometries, detailed geometries and textures. Different features are extracted and analyzed to get the realistic interpretation of different objects. For the primitive geometry, the 2.5D geometry is interpreted by identifying the junctions, edges and faces, and then analyzing the extracted topology. Based on the reconstructed geometry, the geometry of the detailed structure can be achieved by analyzing the shape features of strokes and the relative position, given the architecture style. By drawing strokes of texture, the user is able to get the best matched texture samples from the database, and then map the texture to corresponding surfaces. Finally the realistic looking texture-mapped model is achieved with lighting and background.2. Sketch-based tree modeling. A new method is proposed to generate full 3D tree model which is both complex and realistic looking from a 2D sketch. The user only simply draws several strokes of the branches, and optionally the crown shape. Our system automatically generates the 3D tree model from the drawn sketch. By using the Markov Random Filed (MRF) to model the branch interaction and optimizing the depths of the branches, the 3D model of the drawn branches are reconstructed. The relationship between branches are modeled by the geometry transformation of branches of two successive generations. We also build a database containing several typical tree exemplars. By comparing the 2D branch and crown shapes of the drawn sketch and tree exemplars in the database under different views, the tree parameters of the best matched tree exemplar is achieved and used to guide the reconstruction. New branches are added based on the self-similarity to increase the complexity of the tree model before finally population all branches with leaves. The user is allowed to draw strokes of the crown shape to constrain the branch propagation. Using our system, the user is able to easily create various complex and realistic-looking tree models from sketching.Sketching is a natural and effective way to represent the design concept. Therefore the sketch-based modeling system largely reduces the designer's heavy burden to generate realistic representation, especially in the first stage of design. The concept of sketching reality aims to combine various prior knowledge, model the geometry relationships of the drawn objects, optimize the unknown variables and finally get the realistic interpretation from sketches while reducing the user interaction as much as possible. We successfully applied this concept to architectural design and tree modeling, which proved the effectiveness and flexibility of sketching reality.