Research On The Key Technologies Of Depth Coding For Free Viewpoint Video

With the development of3D display and interactive multimedia systems, new3Dvideo applications, such as free viewpoint video (FVV), is attracting significantinterests. It could provide a more natural and live visual experience in immersionand3D sense than the traditional2D display, which let user choose a range ofarbitrary viewpoint freely. Restricted to the limited transmission bandwidth, we cannot transmit video of all viewpoints. In order to enable these new applications, newdata formats including classical2D video sequences and corresponding depth mapsfor novel virtual view rendering have been proposed. Compared to conventional2Dvideo images, depth maps have very different characteristics. Use of existing codecsto compress depth maps will introduce distortions into the novel virtual views.Therefore, depth coding technologies as current research hotspot, have attractedincreasing researchers inside and outside country. The major work of this paper is asfollows:Inaccuracy depth estimation may influence on depth coding and virtual viewrendering in the free-viewpoint television (FTV) system, an algorithm of multi-viewdepth estimation is proposed to solve the problem for coding and view synthesis.Firstly, check the consistency of initial depth, and the influence of initialmiss-matches is minimized by introduction of an additional adaptive matching errorselection that penalizes the unreliable matches. Then according to certain criteria, themulti-reference depth maps are merged into one disparity map to improve the qualityof disparity map. Finally, a multilateral filtering is used to preserve details in thedepth map and simultaneously smooth the depths in occluded areas at objectboundary, less texture and discontinuity regions. Experimental results show asignificant improvement of the initial input depth maps and coding efficiency, aswell as a reduction of view synthesis artifacts. A depth map compression method oriented to virtual view rendering is proposedto reduce distortion of virtual view synthesis in a free-viewpoint television system.The distortion is caused by quantization of depth data and inaccuracy of depthestimation algorithm. According to the statistics of depth data, a depth down/upsampling techniques based on the non-uniform B-spline curve is proposed. Thedepth map is down-sampled and coded with multiview video coding (MVC). In thedecoding side, the depth map is up-sampled. To deal with the serration phenomenonin edges of the decoded depth image, a bilateral filtering is used to align the objectboundaries. The experimental results evaluated by objective and subjective methodsindicate that the proposed method can reduce bit-rate of depth coding and achievebetter rendering quality. It can improve the depth coding transmission performanceof FVV systems.Since compressing depth map using existing video coding techniques yieldsunacceptable distortions while rendering virtual views, the depth maps need to becompressed in a way that minimizes distortions in the rendered views. We propose adistortion model that approximates rendering distortions caused by depth changes indepth coding. First, we derive relationships between distortions in coded depth mapand rendered view. Then, a rendering distortion model is proposed based on regionalvideo characteristics. Finally, we introduce the new distortion metric into the ratedistortion (RD) model of depth coding instead of the depth map distortion, in orderto reduce the rendering distortions. Simulation results illustrate that the proposedtechniques improve the objective quality of the rendered virtual views by up to2dBover the Lagrange Optimization based mode selection which only considers thedepth map distortions. It can get high-accurate reconstructed images at the receiverof FTV systems.To reduce the complexity of multiview video plus depth(MVD) coding, a fasttexture-depth joint coding scheme is proposed. It makes use of the similaritybetween texture and depth video to skip some unnecessary modes. The algorithm includes following steps: fast mode size decision and adaptive disparity estimationbased on depth information in texture coding, motion vector sharing and early SKIPmode decision by using the correlation of texture image in depth coding.Experimental results demonstrate that the proposed algorithm can reduce thecomputational complexity significantly, without decreasing the coding efficiency ofMVD, which can improve the practicability of FVV systems.