Research On View-Synthesis-based 3D Video Coding

The depth-based 3D TV system is a hot topic in the field of video processing and compressing. The depth-based 3D video format includes depth video which contains 3D geometry information. Such video format could be utilized to generate image of virtual views by view synthesis technol-ogy, which involves the texture and depth image of reference views. Audiences could get a better stereo pair by selecting from the generated virtual views and coded views, thus achieving better 3D perceptual quality. The combination of virtual views and coded views could also provide inputs for multiview display by transmitting only a small number of views of texture and depth sequence, thus presenting higher representation efficiency. Furthermore, the compression of the depth-based 3D video format has been standardized by the international standard organization ITU and MPEG, and this format will be the formal format of the next generation 3D TV system.View synthesis prediction (VSP) is an image predictive coding tool in 3D TV system. The texture and depth images of coded views are used to synthesize a predicting image of current coding texture, thus providing a reference picture with high-level similarity and could elevate the prediction efficiency of texture images.The integration of view synthesis technology into 3D video compression system involves several aspects:1) how the VSP could efficiently generate high quality prediction images,2) how the VSP mode be incorporated in the syntax structure of the 3D video coding system, and 3) the affection of synthesis parameters on the prediction efficiency of VSP.On prediction images generation, block-based view synthesis and disparity picture synthesis using forward warping are proposed. The proposed method significantly compensates the coding efficiency penalty under texture-first coding order with high quality prediction images. Complexity optimization strategies are proposed based on different application characteristics of texture and depth image in predictive coding, respectively. Besides, the proposed methods support both coding order, which provide better universality.On prediction mode design, a VSP-mode-related mode optimization scheme is proposed. The proposed scheme inserts single VSP mode into the merge candidate list adaptively, thus removes the redundancy in the original design.On affection of synthesis parameters on predictive coding, an analytical model on prediction quality considering several crucial parameters is proposed. The proposed model could provide theoretical guidance for parameter determination, and it helps for control of prediction quality and computational complexity in a theoretical way.In general, this thesis is oriented at VSP-based predictive coding for 3D video. The introduced comprehensive research covers synthesis quality, prediction efficiency, complexity optimization, mode design and theoretical performance analysis. A compact theoretical framework of VSP is presented which addresses several key issues of application of VSP in 3D video compression sys-tem. The overall coding efficiency is improved by the proposed work.