Image Structure Analysis In Augmented Reality And Image Stylization

Due to the ubiquity of image capture devices and the development of image-based computer graphics applications in recent decades, the analysis of image structures, such as gradient fields and object contours, is becoming one of the hottest research directions in image processing. The long-term goal of image structure analysis is to build a higher-order description of the image beyond its2D pixel array, and further provide a powerful tool for computer graphics applica-tions. For instance, the use of extracted lines enhances the visual legibility of virtual objects in augmented reality applications, and the employment of gradient fields emphasizes the spatial coherence in image stylization.The main contributions of the dissertation are as follows:1. To facilitate the layout and maintenance of urban underground pipelines, a new approach of photorealistic visualization for underground pipelines integrated into the video of real scene is presented. We exploit section cutaways perpendicular to the ground plane and tunnel cutaways along pipelines to reveal the underground pipelines. To exhibit the correct spatial relationship of the underground pipelines with the objects overground, we segment the screen space into four regions with different depth semantics. We then retain the important visual features of both un-derground pipelines and the overground objects within the particular region for depth cuing and utilize gradual mask for composition. Our approach does not rely on the geometric reconstruc-tion of the3D environment, and its GPU implementation achieves a real-time performance.2. We introduce an importance-driven composition that integrates multiple rendering styles. An importance map, either issued from saliency estimation or designed by a user, is introduced both in the creation of the multiple styles and in the final composition. Our approach accom-modates a variety of stylization techniques, such as color desaturation, line drawing, blurring, edge-preserving smoothing and contrast enhancement. We illustrate the versatility of the pro-posed approach on different applications such as images, videos,3D scenes and even mixed reality.3. We introduce a new approach for defining continuous non-oriented gradient fields from discrete inputs. Our approach builds on a moving least square formalism (MLS) where the key is the computation of higher-order local approximations of non-oriented input gradients. In particular, we show that our novel isotropic linear approximation outperforms its lower-order alternative (a continuous extended version of structure tensor):surface or image structures are much better preserved, and instabilities are significantly reduced. Its ease of implementation (on both CPU and GPU), and its small performance overhead will find a widespread use in graphics applications.4. We further demonstrate how the quality of image and video stylization can be improved based on the improvement of MLS gradient field. The stylized results have controllable smooth-ness without losing the image structure of the original image. Besides the spatial coherence, we also preserve the temporal coherence for line extraction using profile analysis of image features, which are achieved by a1D polynomial fitting along the gradient direction. Profile attributes can be used to distinguish between sharp and smooth features, and quantized profile parameters are then mapped to stylistic parameters such as brush orientation, size or opacity to give rise to a wide range of line-based styles.This dissertation focuses on the analysis of image structures, and the improvement that ben-efits augmented reality and image stylization applications. Though each chapter is independent, they are connected with this main topic progressively. The conclusion and promising further research directions are discussed in the final chapter.