Research On IBR Technology Oriented To The Multi-view System

With the development in technology of computer and communication, it's believed that the three-dimension video display will be the next natural and logical development in the video display society, which could provide a more natural and live visual experience in immersion and three-dimensional sense than the traditional two-dimension display. FVV (Free Viewpoint Video) or FTV (Free Viewpoint Television) is the main application of three-dimension video. Other than the three-dimensional sense, another characteristic of FTV is interactivity, which let user choose the viewpoint freely. Restricted to the limited transmission bandwidth, we cannot transmit video of all viewpoints. The common solution is the IBR (Image-Based Rendering) technology. It generates video of new viewpoint only by some reference views, which can reduce the data needed for storage and transmission. The major work of this paper is as follows:A method for multi-view synthesis based on the Tikhonov regularization is proposed. It uses the bi-direction PDE to iteratively optimize the disparities, which benefits to enforcing the smoothness while preserving the edge information of the disparity map. Such optimization is initialized by the disparities obtained from our centralized phase-based support-weights (CPBSW) approach. Experimental results verify the validity of our proposed method.For the free viewpoint television system, a sparse Ray-Space interpolation method based on hierarchical prediction is proposed. Firstly, disparity is estimated and optimized by our proposed bi-direction PDE method, then hierarchical prediction structure is proposed to interpolate the virtual views. Experimental results show that both the performance and the robustness of the proposed algorithm are much higher than the traditional block matching based interpolation (BMI) and the inter-slice correlation based method. To reduce the transmit bandwidth while preserve the quality of reconstructed virtual views, a novel image-inpainting-based DIBR method is proposed. It employs depth image pre-processing and image inpainting method to fill up the holes after 3D image warping, which could synthesized the new viewpoint only by one reference viewpoint. It means only video in one reference viewpoint is needed to be transport in our method, and much more bandwidth could be saved since traditional method needs to transmit video in at least two reference viewpoints. Experimental results verify the validity of our proposed method which can generate arbitrary viewpoint for real scene.Reconstruction of intermediate view from arbitrary view plays an important role in the FVV system. To get a better rendering result, we proposed a TLS (Total Least Square)-based method to remove the mismatch and noise produced by Depth Estimation Reference Software in the background area of disparity map. Experimental results show considerable improvement on the quality of synthesized image.Lightfield rendering is an important IBR technique. A lightfield rendering method based on region of interest is provided, which can synthesize a pleasing result according to the camera parameters and geometry of the scene.