Study On Virtual Viewpoint Rendering Algorithm Based On Depth Image

As a new video technology, Free Viewpoint Video (FTV) which can providestereoscopic effect allows the users according to their own will to realize thereproduction of3D scene without any auxiliary apparatus and gives people a kind ofbrand-new visual feeling as if traveling in the pictorial world by means of realistic3Dsense of space, therefore it has epoch-making significance for the development of thevideo technology. Virtual viewpoint image rendering which is considered as a keytechnology for free viewpoint video can restore information in different viewpoints forthe same3D scene under the restricted conditions of storage space and transmissionbandwidth, and it can provide strong technical support for transforming viewpointinteractively, freely and smoothly.The virtual viewpoint rendering technology can be roughly divided into two kinds:Model Based Rendering (MBR) and Image Based Rendering (IBR). MBR is notsuitable for processing higher complexity natural scene bacause the amount ofcomputation is related closely to the complexity of the scene. Compared with the MBR,IBR does not demand of the complexity of the scene too much and can generate theimage with strong sense of reality. Depth Image Based Rendering (DIBR) brings depthinformation into the IBR that can largely save the data storage space and transmissionbandwidth, so it is more practical than other pure IBR technologies, but the problemslike hole, artificial contour, overlapping and resampling will appear due to the factorssuch as occlusion and inaccurate depth image when rendering virtual viewpoint imageby DIBR, which seriously affect the visual effect of the rendered image.This paper is based on the Depth Image Based Rendering technology, theviewpoint rendering process is divided into two parts, namely virtual viewpoint cameraparameters estimation and image rendering, and have been subjected to intensive studies,respectively. The research contents include: the realization of virtual viewpoint cameraparameters estimation and virtual viewpoint image rendering based on half pixelprecision.The main work of this paper is as follows:(1) Three algorithms of virtual viewpoint camera parameters estimation areproposed, which are the virtual viewpoint camera parameters estimation algorithmbased on relationship modeling, the virtual viewpoint camera parameters estimationalgorithm based on the difference between feature points and the virtual viewpointcamera parameters estimation algorithm based on linear proportional relationship. According to the camera parameters of the reference viewpoints, the mathematicalrelationships such as the difference between feature points on the different viewpointsand the relative positions are used to estimate the camera parameters of the virtualviewpoint, and the virtual viewpoint images are rendered by using the reasonablyestimated camera parameters to improve coherence of the rendered images and allowthe viewers to get more realistic and coherent visual effect.(2) A virtual viewpoint image rendering algorithm based on half pixel precision isproposed. Firstly the reference images are preprocessed by half pixel precision toimprove the rendering accuracy; then forward3D image warping is utilized to generatethe image and depth image of the virtual viewpoint, inverse3D image warping is usedto fill part of holes caused by some factors such as inaccurate depth information;artificial contour is erased according to depth comparison and the rendered images areprocessed by brightness adjustment to eliminate brightness discontinuity problemaccording to simplified model of the brightness differences between images; Finallydifferent methods are used to fill the remaining holes according to the types of the holes.The experimental results show that the proposed method can obtain satisfactoryrendering results.