Research On Weathering Simulation And Illumination Consistency In Mixed Reality

Along with the rapid development of computing technology, the subject on how to seamlessly synthesis virtual and reality world together has drawn wide range of attentions. After many years of development, researchers have made important progress on the problem of geometry consistency in mixed reality. We can recover camera parameters and3D geometry of real scenes at real time, registering virtual objects into the video stream correctly. The up-to-date researches mainly focus on how to solve consistency on illumination and interactions between virtual and real objects. The key purpose is to improve visual consistency between virtual and reality scenes, and to provide more convenient and efficient interactive tools.For these reasons,this thesis will focus on the problems of weathered appearance simulation and illumination consistency in mixed reality. Several methods on weathering models, interactive weathered appearance editing and lighting model registration are proposed separately. With these techniques,high-quality augmented reality effect can be achieved. Our work’s as follows:We present a general weathering simulation framework that is effective for modeling a wide variety of weathering effects on virtual objects. Our technique is based on a type of aging-inducing particles called γ-tons which are never mentioned before. With ［-ton tracing through the scene controlled by the user-defined weathering transport rules, we can produce weathering effects that are customized to both the virtual object geometry and user’s will. Besides, several effects, such as stain-bleeding,multi-weathering, and large-scale geometry changes that are challenging for existing techniques can be readily captured by our method, expanding the capability of handling different kinds of weathering processes.We propose a novel interactive weathering method for real-captured videos, which can in-teractively synthesis weathering effects onto video objects. We first recover the depth maps of the input video. In order to maintain the temporal coherence,we employ employ a new depth-based interactive video object cutout method to efficiently extract the target objects we interest. Then, we propose a novel point cloud sampling method to sample the3D point cloud from the recovered depth maps which can robustly reject outliers and simultaneously control point redundancy, and then use the point cloud to reconstruct the3D geometry mod-el.With the reconstructed3D model,γ-ton tracing is introduced into the pipeline to create the weathering map used to edit the video object. The problem of γ-ton tracing is that it does not provide any interactive visual feedback.Therefore, as an improvement, our system also provides two interactive weathering tools, which allows the user to specify the weathering area or tune the weathering speed to achieve the desired effects. With these interactive tools, weathering editing can be more flexible and much easier.We propose a feature-based alignment algorithm, which can automatically align the illumi-nation environment map with the captured real scene. Affine-SIFT is employed to extract and match the common features between the environment map and the captured video. RANSAC method is used to remove the outliers. Then the3D positions of the matched feature points can be computed by structure-from-motion technique. Finally.according to these3D match-es. the correlation between the environment map and the real scene can be obtained and the automatic alignment is accomplished. With this method,we can register the lighting informa-tion to the scene and the illumination consistency can be guaranteed.Based on this technique, we also present a high-quality real-time augmented reality system. The proposed system em-ploys a key-frame based real-time camera tracking technique to robustly register the virtual objects into the online video stream. The users are allowed to real-time edit the realistically inser(?)d virtual objects. During the rendering pass, with the auto-aligned illumination en-vironment, we use the importance sampling algorithm and shadow mapping technique to achieve high-quality seamless mixture of virtual and real objects.