Study On Multiple View Geometry In Augmented Reality

Multiple View Geometry in Computer Vision has build an all-around and sophisticmapping model between shapes from projective objects and camera's parameters. Thismodel is composed of many geometry knowledge, particularly the complex and grace-ful geometric relationship among several views. It has also become one of the mostimportant fundamental theories of understanding real world for computers. This dis-sertation studies the application of Multiple View Geometry in Augmented Reality, andmain contributions are:(1) A new vision parameter estimation method using Data Uncertainty-basedConstrain Resolver Model (DUCRM): Data describes the characteristics of real objects.It also carries the uncertainty which is not wanted. In most of current estimation meth-ods, data uncertainty is overlooked and simplified. The accuracy of estimated param-eters thus is affected. This dissertation defines the analytic representation of data un-certainty on image and provides the algebraic equations for uncertainty propagation innonlinear functions, which are all based on data uncertainty theory. It provides a four-layer approach and a revising scheme in minimization iterative process. The accuracyof estimated parameters are thus improved.(2) Improved planar homography estimation algorithm using mixed operatorand improved epipolar geometry estimation algorithm based on constant elimina-tion: Calculating planar homography between two low resolution images is always adifficult problem because of larger errors in comer detection and matching. We pro-vides a mixed detection algorithm using Harris comer and curve functions to achievebetter results. And with DUCRM, we get a better result compared with other methods.In addition, Epipolar Geometry is also an important vision parameter for perspectiveprojective applications. Traditional linear methods is simple but subject to noise per-turbance. And nonlinear methods give better result by a global searching, which iscomputing expensive. Experiment shows that simply using DUCRM leads to singularproblems which cause numerical calculation unstable. We provide an approach by elim-inating the constant column in measurement function. Results show that the singularmatrix is avoided and computing speed is accelerated.(3) An accelerated algorithm of occlusion handling between virtual and real ob-jects: Correct, fast and robust occlusion handling between virtual and real objects isalways a critical problem in interactive AR. How to accelerate the computing speed is the key of using depth map based methods. We provides an approximation methodwhich can be implemented on PCs. First, a global and local Gaussian kernel function isused to extract foreground objects. And then, stereo matching is conducted only usingpoints on objects' contour. This process is also accelerated by Epipolar Geometry. Fore-ground object's depth is thus estimated using triangle method and occlusion is detectedby the overlapped region. Compared with other methods, ours is fast and gives betterresults. However, since one object has only one depth value, this method can not beapplied to deformable objects.(4) A fast shadow rendering algorithm in dynamic scene based on reflective lightball: Although BRDF and Image-based Relighting methods have good models to re-cover light sources, they can not done in real time as for a dynamic scene. We providesa discrete light sample method to approximate the light source in real scene, which us-ing a light reflection ball. This method is applied in a Video-based Augmented Reality(VAR) system simulated by PCs without GPU acceleration.Based on the estimation model and algorithms described above, we also developedan interactive Video-based Augmented Reality prototype. It is a mixed environmentwhich provides tools to enable observers walk through and interact with virtual seaanimals. In this prototype, algorithms are implemented and tested. And interactivescheme between observers and virtual avatars is also designed and implemented. Thisprototype helps us explore the feasibility of creating and exhibiting such systems inScience Centers.