| Compared with the conventional single channel video, the amount of data of free-viewpoint videos to be processed is at least double, generally several times against a conventional single-channel video. Since people improve the requirement of video quality, high resolution and high definition are necessary in future, which will also bring a sharp increase in the amount of video data. As for720P format, to achieve practical video quality and25frames-per-second display rate, the video containing one viewpoint video could generate2700kb/s (about1800MB/h) data with traditional2D video coding standard, as well as3600MB/hour data for two viewpoint stereo video. It cannot resolve the problems result from the data by raising the amount of storage and network bandwidth. It must adopt a more efficient data compression coding technology in order to store and transmit free viewpoint video effectively.MPEG proposed an encoding method for multi-view video plus depth video format. It focuses on FVV and3DTV systems from the perspective of specification, and includes expression generation, processing, rendering of MVD data. In applications of free viewpoint television, multi-view format plus depth significantly reduced the viewpoints need to be coded and transmitted. In aid of the depth image-based rendering, the target virtual viewpoint video is derived from decoding reference viewpoint video. The techniques provide users with good3D visual and interaction experience, while save the bandwidth significantly.In MVD coding system,3D perceptual coding is the main stream to further improve the encoding efficiency of color videos. Based on maintaining the backward compatibility for MVD coding standards, in-loop filtering based hybrid coding framework is raised to improve the quality of coding depth video, and the hole caused from occlusion is filled for enhancing the quality of color video in the synthesis of virtual viewpoint. The depth image is used to represent distance information for a pixel in the scene, which could effectively describe three-dimensional geometry information of the scene and assist rendering the virtual view in an arbitrary viewpoint. This paper focuses on how to solve the below problems.The National Natural Science Foundation of China (No.61271256) and Microsoft Research Asia Innovation Project (No. FY09-RES-OPP-013) are the foundations of our research in this paper. The binocular stereo perceptual mechanism is analyzed, and the impact of binocular perception and saliency factors on masking effect is taken into account in3D JND model. The3D JND model is adopted to enhance the efficiency of coding stereoscopic by residual filtering. The research about depth image filtering and hole filling in color video is funded by the National Natural Science Foundation of China Youth Project (No.61003184), and the National NSFC (No.60970160).Specifically, this paper will research the stereoscopic perceptual video coding, depth image-based rendering for virtual viewpoint generation, and virtual view rendering-based image restoration. The proposed methods reduce the number of bit rate cost by removing visual redundancy. Thus, the methods could provide high-quality stereoscopic video at the same condition of bit rate by improving coding efficiency. The main contributions are belows:(1) Depth contrast-based3D JND modelingSince the current3D JND models are based on traditional2D physical and psychological perception, which ignores the masking effect between binocular eyes, this dissertation proposes3D JND model by integrating of depth contrast factor from the perspective of stereoscopic perception mechanism, as well as take the model to guide the residual filtering. The method further improves coding efficiency for stereoscopic video by removing human visual redundancy.(2) Depth map filtering algorithm based on the corresponding characteristics of the color imageThe existing depth map filtering methods based on color image don’t give an accurate description of the coherence characteristics between the color image and the depth image. The texture interfere the accuracy of the filter coefficients in non-textured regions of depth image, which results in blurred edges and jagged effect in a filled region. To solve this problem, this dissertation proposes LBP-based description model of edge features for texture image, and then classifies the surrounding pixels with the center pixel by texture LBP-based description in edge regions of depth image to improve the quality of filtering depth map.(3) Virtual viewpoint image restoration algorithm based on the relevant characteristics of the depth imageThe texture image restoration methods based on depth image depend on the constraint of depth continuity, so ignore detail information of texture image. While repair the hole for complex scenarios, it leads to blurring and blocking effects. The existing methods also copy the blocks from neighbor regions, which cannot describe non-rigid motion in color image, and result in discontinuities between adjacent remaining regions and filled regions and severe artificial effects. To this end, the paper takes hole-filling as a reverse texture synthesis in color image, and model the fusion of depth information and texture detail texture. Under the constraints of texture detail and depth map, the best matches are searched from global regions. The best matches are fused to generate the hole-filled content. Therefore, the texture consistency is enhanced and non-rigid motion is measured. In summary, this dissertation calculates a depth masking-based3D JND model (Just Noticeable Distortion, JND) to realize residual filtering for stereoscopic video coding. The residual filter reduces the coding bitrate by removing human visual redundancy. According to the structural similarity between the color and depth image, the depth filter achieves noise reducing for depth decoding video, and the image restoration algorithm recovered the hole in rendered color image. The proposed methods improve the efficiency of coding stereo video and quality of the virtual viewpoint video. |
PDF Full Text Request