Multiview Video Coding And Virtual View Rendering

Multi-view video, which can provide viewers with the benefits of added realism, selective viewing, and improved scene understanding, will be widely used in the fields of 3D TV, free-viewpoint TV, immersive videoconferencing, virtual reality etc.One major obstacle for using multi-view video is the large amount of data. A multi-fold increase in bandwidth over the existing single-view makes it extremely tough to transmit and store multi-view video data. To achieve high compression efficiency, co-relation between the different views must be exploited in multi-view video coding scheme. We uses advanced predictive methods to eliminate the spatial, temporal and inter-view co-relation in multi-view video. According to the characteristic of multi-view video, global disparity coding method is employed. To decrease the coding bit number of motion vector (MV) and disparity vector (DV), an optimized MV&DV predicted method is proposed. It utilizes not only the MV&DV of neighboring blocks, but also that of corresponding blocks in the adjacent images. Due to the distinct performance of cameras, there exists serious color-variation (including brightness variation) between the images of different views. These variations will impact the compression performance of multi-view video coding. To improve the coding efficiency of MVC, we investigate the method of color-variation compensation disparity prediction. Firstly we analyze and model color-variation based on the image formation theory. Then, various color-variation compensation methods are proposed according to the simplified models. They are global linear color-variation compensation, global non-linear color-variation compensation, local color-variation compensation and global-local-adaptive color-variation compensation.Different from mono-view video coding, disparity estimation (DE) is also utilized to reduce inter-view redundancy in MVC. Taking coding efficiency into consideration, block-based DE, like motion estimation (ME), is more appropriate for MVC because it has better compatibility with the existing video coding standards. Consequently, the prediction part, which consists of ME and DE, becomes the most computationally intensive part in an MVC system. Therefore, ultra high computational complexity is a critical design challenge for MVC. Fast macroblock mode selection algorithm can be used to accelerate the encoding speed for MVC. The fast algorithms for single-view video coding cannot be used directly for MVC because the prediction structures for MVC are different from those of single-view video coding. In this paper, a hybrid fast macroblock mode selection algorithm is developed for MVC.There are mainly two different approaches to generate arbitrary viewpoint images. One is the Model-Based Rendering (MBR) and this method consists of creating a 3D Computer Graphics (CG) model of the object inside the image. However, it is difficult to make a graphic model close to the real object. As a result, the created image appears to be artificial and unnatural. On the contrary, there is another method called Image-Based Rendering (IBR), which is not related to the geometrical characteristics of the object and can generate a photorealistic image from captured images. In this paper, two IBR methods are developed for view generating; one is for ray-space data and the other use 'depth + color' data structure.