Research On Image-based Virtual Walkthrough

Image-based rendering (IBR) is widely used in virtual walkthrough in real-time because of its simplicity and efficiency. Image registration and mosaics are the basis techniques of IBR which are also widely used for a variety of tasks in computer vision and real-time computer graphics , including virtual environments, panoramic photography and video compression. This dissertation aims at constructing polygonal panoramas based on simple equipment and accurate image registration to realize virtual walkthrough in real time. The main contributions of this dissertation are outlined as follows:(1) A new method for constructing and refinement of panoramic mosaics from sequences of images gotten by hand-held cameras is proposed. First, we take initial registration with the phase correlation based on Fourier-Mellin transforms that enhance the robustness of registration. Second, we improve the registration precision by calculating the parameters of the projection transform with an iterative optimization that includes additional illumination model. Compared with other image registration algorithms, this method can improve the success rate of the automatic registration and mosaics, especially when the hand-held camera has an irregular movement or adjacent images have differences of brightness.(2) A global optimization approach for construction of a 360°full-view polygonal panoramic mosaic is presented with global consistency. There are three primary steps in this approach. The first is a local registration using SIFT. In the second step, an efficient algorithm for finding a global consistent registration of all images is developed. The local registration matrices with the boundary condition are used for computing the initial parameters of the iterative optimization for the global consistent registration, to adjust/reduce the alignment error accumulated by the local registering and stitching of adjacent two images. Lastly, the overlapping areas of the images are stitched by the graph cut algorithm and blended with a multi-resolution method. It is proven by the experiment results that the presented approach can effectively construct polygonal panoramic mosaics with the global end-to-end consistency.(3) A new method of virtual walkthrough using image-based rendering with the polygonal panoramic mosaics is presented. First, we take pictures and get the polygonal panoramas with a hand-hold camera along a route path or in a polygonal region. Second, we use SIFT for the feather point detection to get slim images with depths which can be used for the interpolation for the rendering at a new view point. The algorithm proposed is very efficient for virtual walkthrough in a large outdoor scene. The construction and rendering process with the polygonal panoramas is easy to implement and the data capturing is simple without any expensive equipment.