Research On Registration Between Virtual And Real Worlds In Augmented Reality Systems

Augmented Reality (AR) is a growing area in virtual reality research. An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information. The augmented reality presented to the user enhances that person's performance in and perception of the world. The ultimate goal is to create a system such that the user can not tell the difference between the real world and the virtual augmentation of it. To the user of this ultimate system it would appear that he is looking at a single real scene.Registration is one of the most pivotal problems currently limiting AR applications. It means that the virtual scenes generated by computers must be aligned with the real world seamlessly. Computer vision techniques have the potential to provide the accurate registration data needed by AR systems. The Vision-based methods do not require any special and expensive equipment except for common cameras and a personal computer. These kinds of methods take full advantage of scene features including man made markers, feature points, planes, and so on, to achieve registration between real and virtual scenes. In this paper, we especially focus on the computer vision based registration approaches. We mainly investigate the uncalibrate marker based method, marker and natural feature based method and multi-planar structures based method to improve the usability and robustness of the current AR systems.The main studies and achievements of thesis are listed as following:(1) Marker based uncalibrate registration methodWe propose a novel online dynamic calibration method to solve the problem that the intrinsic parameters of the camera can not be changed in the conventional marker based registration systems. Our method can calculate the intrinsic parameters according to the projections of the four corners in online stage and improves the usability and accuracy of the man made marker based AR systems to a large degree.(2) Registration under the circumstance of partially occlusion in marker based AR systemsWe present a new registration method based on global homography which can avoid the problem of occlusion and error accumulation in marker based Augmented Reality systems. We take following two steps to ensure the availability of our method. Firstly, a new affine invariant feature: edge-corner, is introduced to provide a robust and consistent matching primitives. Secondly, global Homography is used to calculate transformation matrix robustly by matching current and first frame to avoid error accumulation problem.(3) Registration using marker and planar structuresWe also propose a marker and planar texture based registration method. The initial registration matrix are obtained using known-size marker. The current registration matrix are calculated using homography between the current and previous frames when marker is occluded.Experiment results illuminate that our method improves the robustness and practicability of marker based registration systems particular under the circumstance of severe occlusion of man made markers.(4) Multi-Planar structures based registration for Augmented Reality systemsWe illustrate a novel registration method for augmented reality systems based on multi-planar structures and natural features tracking. The method can be divided into two stages: offline 3D reconstruction and online registration. In offline stage, the planes and world coordinate systems are established based on 3D reconstruction technique firstly. Secondly, we train the system by synthesizing a large number of views of matched features and by using PCA and K-means to generate a compact description of this view set. In online stage, we rely on this description to fulfill the task of feature matching based on nearest neighbor search method. With the obtained feature correspondences, we compute the homographies between the world planes and the current frame robustly using RANSAC and nonlinear least square method. Then, the registration matrix is synthesized using the above homographies and the transform matrices between the planes and world coordinate system obtained in offline stage. Our method cast off the requirement that the initial camera position should close to the reference images, while overcoming the problem of error accumulation. Some experiments have been carried out to demonstrate the validity of the proposed approach.