Research Of Arbitrary Virtual Viewpoint Image Synthesis Based On Two Images

With the rapid development of computer vision etc, the gross domestic product(GDP) of virtual viewpoint image synthesis technology applied in these industries continuously increases, such as film and television, medical diagnosis, transportation and security, etc. The virtual viewpoint image synthesis technology is a technique using a finite number of cameras distributed in different viewpoints to shot the same scene, and a new image is synthetic from two or more original images by digital processing which can describe the objects from multi-angle and comprehensive.After surveying most of the existing virtual viewpoint image synthesis technology home and abroad, it is found that some problems still exist in virtual viewpoint image synthesis, such as the synthesis precision is poor and the movement range of virtual viewpoint is limited. On the one hand, the virtual viewpoint image synthesis based on 3D reconstruction requires the complicated camera calibration, and calculate the three-dimensional coordinate in the space of all the corresponding points on the images accurately. Any error calibration will lead to failure of modeling, so this method has high convexity, low accuracy, poor flexibility and real-time; On the other hand, the virtual viewpoint images don’t accord with projective geometry relationship and have serious distortion by some traditional virtual viewpoint image synthesis based on image morphing. Seitz developed the image morphing method and solved the problem by proposing the view morphing method, however, the movement range of the virtual viewpoint is still limited on the line between the two input viewpoints.In this paper, the synthesis precision of the arbitrary virtual viewpoint image and the movement range of virtual viewpoint are mainly studied. We propose a novel and effective method according to the Image-based Rendering(IBR) and the intrinsic projective geometry between the two input images to synthesize a new arbitrary virtual viewpoint image. The simplified model of the viewpoint images is built in a suitable coordinates system to analyze the projective geometry relationship of the corresponding points between the input viewpoint images and the arbitrary virtual viewpoint image. The each image is handled by the process of feature point extraction, correspondence matching, view parallel, triangular segmentation, texture matching, perspective projection transformation. An arbitrary virtual view image can be generated in the space only using the positional ratio parameters between the input viewpoints and virtual viewpoint. Using this method, complex three-dimensional reconstruction and camera calibration can be avoided, and the limitation of the movement range of the virtual viewpoint based on traditional View Morphing technology can also be solved. At the same time, the unnatural phenomena caused by the light and shade is resolved using the linear interpolation technology. A technique to estimate the error range of the arbitrary virtual viewpoint image was also developed by computer simulation, and do the experiments by comparing the synthetic images with the real shooting images.The experiments and simulation results show that the arbitrary virtual viewpoint images can be obtained only using the positional ratio parameters between the input viewpoints and virtual viewpoint through the method proposed in this paper, even if the two input images can’t be parallel completely. The proposed method is reasonable and effective.