Research On Seamless Fusion Of Real And Virtual Scenes In Augmented Reality System

Augmented reality (AR) is a new developing technology based on Virtual Reality(VR) but expand VR, and AR enhances the real world by superimposing computer generated information on top of it. AR relates to a good many fields of computer science. The early researches on AR mainly focus on Registered in 3-D, that is, its main object is to improve the tracking precision and the veracity of matching real and virtual scenes. Recently, with the development of research, it has become the new research hotspot that how to improve the visual consistency of synthetic images and how to produce more realistic synthetic interfaces.The problem of seamless fusion in augmented reality includes two aspects, the one is geometric consistency, and the other is illumination consistency. Aim at improving the third dimension of synthetic images in AR, this dissertation does some in-depth investigations on the above two aspects. The main contributions of the dissertation are as follows:1,Tracking and registration are two key technologies in AR, and their precision will directly affect the superposition of real and virtual scenes. However, the measurement error or calculation error always exist in the two processes, which calculate the transformation of coordinates and track the view point of user. So in the dissertation, we firstly give the perturbation analysis of calculating the transformation of coordinates, and then a selection criterion of initial metrical data is put forward, consequently to minish the calculation error. In addition, as for the nonlinear measurement error of magnetic tracker, we propose to rectify it using improved BP (Back Propagation) neural network in the dissertation.2,With the improvement of the whole system and considering the tracking range and precision, we subsequently added stereo-vision sensors. So in the dissertation, a data fusion model based on the notion of fuzzy integral is presented to combine the results of multiple tracking sensors. According to the application character in AR, the tracking range and the tracking error have been chosen to act as evaluation factors. A method for dynamically assigning weighting factors, using the comprehensive performance evaluation of individual sensors is also proposed. The fuzzy integral approach can release the user's burden from tuning the fusion parameters.3,Realistic occlusion phenomenon is an important part of making AR environment convincing, as occlusion is well-known to be a strong depth cue. Mutual occlusion between real and virtual objects enhances the user's impression that virtual objects truly exist in the real world. So it is another important factor of geometric consistency.An algorithm for realizing multilayer occlusion in AR is presented in the dissertation. We have designed a special scene graph tree comprised of some special nodes, that is, EMO nodes. According to the location of real moving object, different EMO node will be activated in real-time, consequently realizing the multilayer occlusion. Differing qualitatively from previous work in AR occlusion, our algorithm realizes multilayer occlusion, and its application domain involves indoor-field occluded objects, which are several meters distant from the viewer. Previous related work has focused on monolayer occlusion, and near-field occluded objects, which are within or just beyond arm's reach.4,For a seamless AR system, apart from the geometric registration technique, the proper simulation of real lighting is another important factor. Such a factor determines the consistency of illumination between the virtual objects and the surrounding real objects. In the dissertation, two spheres with Lambert surface are introduced in the real scene. Through image analysis and analytical geometry of three dimensions, the illumination parameters of the real scene are estimated automatically. These parameters describe a point light source and ambient light. Using the estimated illumination parameters, the synthetic scene is relighted through inserting proper virtual light sources in the scene graph tree. This work explores an image-based analysis and post-illumination-processing approach to rapidly improve illumination consistency in AR system.