Research On3D Video Coding Using Depth-Image-Based Rendering

As people continue to increase their visual appetite and to demand more vivid visual effects of the depicted scene, three-dimension video (3DV) system becomes an important research issue. As compared to the traditional two-dimensional video technology,3DV is able to provide the perspective description of the natural scenes through the additional depth information. Moreover,3DV allows the users to select the view point and view direction within the captured real scene range freely and interactively. Due to these advantages,3DV technology has been widely applied to several fields, such as3DTV, FTV, visual reality, sporting events broadcasting, and visual surveillance. Currently,3DV systems are usually implemented by using depth-image-based rendering (DIBR) technology. A typical3DV system is made up of several important technical aspects, including3D video capturing, scene representation, video data coding, scene rendering, and interactive display. In order to achieve better3D experience, it is necessary to capture multi-view video data for3DV system. Usually, there are other auxiliary informations need to be captured. This inevitably led to the huge amount of video data. However, the network bandwidth resources and the storage resources are often very limited in practical applications. Thus, researches on efficient compression of the3D video data become essential. As a result, it is of theoretical significance and practical value to conduct an in-depth research on3D video coding.In chapter1, the significance of the research work is presented, and the current research status is briefly summarized. Then, the main research contents and the chapter structure of this dissertation are introduced.In chapter2, the research on the reference viewpoints selection of3D video system is carried out, and a rate-distortion based reference viewpoints selection method is proposed. The entire rate distortion model of the display viewpoints is first deduced through the analysis of the compression distortion of the reference viewpoints, synthesized distortion of the virtual viewpoints, and the compression rate of the texture videos and depth maps. Then, the reference viewpoints selection problem is represented with a constrained optimization problem. Finally, the optimal setting of the reference viewpoints for different bit rate constraint is obtained by computing a few model parameters.In chapter3, research on multi-view texture video coding is carried out, and a visual saliency based multi-view texture video coding method is proposed. Firstly, a novel video saliency filter which combines the color and motion information is proposed. Based on the video saliency filter, the visual saliency map of the intermediate viewpoint is obtained. Secondly, the visual saliency maps of the other viewpoints are obtained by3D image warping. Then, the saliency value of each encoding macroblock is computed by using the saliency maps. Finally, with the principle of the perceptual video coding, the encoding quantization step of each macroblock is adaptively controlled by its saliency value. This leds to the adaptive encoding quality control of the macroblocks.In chapter4, research on the depth map coding is carried out, and a just noticeable disparity error based depth map coding method is proposed. First, the just noticeable disparity error (JNDE) of each depth pixel is defined as the maximum tolerable disparity error, such that any depth error inside the JNDE range would not cause the distortion in the synthesized view perceived by human eyes. Then, the JNDE values are applied to the intra and inter prediction process during the depth map coding to reduce the prediction residues. Moreover, the residues are further adjusted by JNDE values to reduce the variances of the residual blocks.The accomplishment and the innovation of the whole research are summarized in the final chapter, and the prospect of the future work is also being proposed.