Research On Depth Map Extracting Of Stereo Video And Depth Sequence Coding

The Multi-view Video is widely used in Three Dimensional Video (3DV) system,Free Viewpoint Video (FVV) system, Three Dimensional Medical Display system andThree Dimensional Video Surveillance system, etc. Free Viewpoint Video is a typicalapplication of Multi-view Video in daily life for the majority of consumers, and FVVsystem becomes a hot topic in the field of video signal processing at this stage. As the newgeneration of visual multimedia systems, FVV allows users to watch a video scene at anyangle, to meet the needs of the sensory experience from different perspectives of the videosignal. Multi-view Video plus Depth (MVD) is the typical data format of Free ViewpointVideo system. The problems of Multi-view signal acquisition, transmission and storageare solved by the virtual viewpoint rendering technology. Virtual viewpoint renderingtechniques can use any two of the multi-view video viewpoint color video sequence andtheir corresponding depth sequence to synthesize an arbitrary position video between thetwo viewpoints video signals, to meet the requirement of watching in any angle. FreeViewpoint Video system based on MVD data representation is consist of Multi-viewpointcolor video collecting, Multi-viewpoint depth sequence extracting, Multi-viewpoint colorvideo sequence encoding, Multi-view depth sequence encoding, color/depth sequencetransmission, Multi-view color and depth sequence decoding, virtual viewpoint renderingand three dimensional display, etc.Multi-view Video technology brought pleasant and convenient into human life in thephysical and psychological sense, meantime, it also faces many technical bottlenecks. Inthe facet of depth map acquisition, the lower resolution and higher cost of Multi-viewdepth sequence acquisition device are limited to collect multi-view depth sequence.Therefore common depth map extraction algorithm is used to obtain the depth map, butthe existing methods produce a certain amount of wrong matching in the texture area,lower texture area and depth discontinuities area in the image. In the facet of depthsequence encoding, due to the architecture of depth map sequence is different from thecolor texture video，there is a greater spatial redundancy in depth map sequence, and thedepth map is the intermediate step oriented to render virtual viewpoint video, the qualityof the decoded reconstructed depth map affects the rendering quality of the virtualviewpoint video seriously. Given the two faced technical bottleneck of depth mapextraction algorithm and depth map sequence compression method, this paper presentsfour algorithms to solve the problems.1. According to the inaccurate problem in the target boundary area and fine texturearea of current matching algorithm, the concept of the normal to the planar luminance isproposed. Through the analysis of luminance normal, grayscale image luminance normalcan reflect the high-frequency information of the depth map. Given the characteristics ofgrayscale image luminance normal, a luminance normal similarity and adaptive weightsmatching algorithm is proposed, which is base on the adaptive weight matching algorithm.The algorithm improves matching precision of the object boundary and fine texture regionin the depth map.2. According to the mismatching problem in the low-texture area of the matching algorithm, a convex function method is proposed to solve the problem. According to thedisparity energy characteristics between the stereo image pair, stereo matching question ismodeled into convex function optimization problem by introducing a secondarysmoothing factor. It searches the optimal solution in the total variational constraint set bythe sub-gradient projection method. This algorithm not only keeps the edges of the imagewell, but also reduces the mismatching of the low-texture area, and improves the matchingaccuracy of low-texture area.3. According to the problem of the artificial effect quantization step introduced by theencoder, especially the problem of leading to wrong edge of the target in the decoding. Amethod is proposed to detect the interest edge of the target in the depth map, thereby toretain the interest edge of the target by adjusting the quantization parameter of the encoder.When the transmission rate is keeping invariant, the new video is rendered by the decodeddepth map, the experimental results show that this method improves the subjective qualityof the virtual viewpoint.4. According to the depth map coding scheme of down/up sample resolution, amulti-similarity and adaptive weight sampling method based on adaptive weight isproposed. Through the analysis of the structural characteristics of the depth map edge,three schemes are proposed to reconstruct the full resolution depth map by combiningfilter and the sampling method. The first scheme is established by combining the keepingedge à-Trous wavelet filtering and adaptive weights sampling method, the second schemecombines median filtering and multi-similarity adaptive weights sampling method, thethird scheme is proposed by combining keeping edge à-Trous wavelet filtering andmulti-similarity adaptive weights sampling method. The three schemes all can keep betterdepth map edge and reconstruct the normal resolution in low bitrates. The virtual view isrendered by the full-resolution depth map sequence reconstructed. The subjective andobjective quality of the virtual video are improved.