Research On Key Technology Of Free-View Video On The Basis Of Depth Image Rendering

Free viewpoint video can provide users a range of arbitrary viewpoint by depth-image based rendering (Depth-image Based Rendering, DIBR), which is expected to be the next generation of digital video standards. The realization of DIBR free viewpoint video is related to the key technologies such as image correction, stereo matching, multi-view video coding and virtual view synthesis. There are many problems in the implementation of these technologies:the scaling and rotation problems between viewpoints is not be considered in the existing image correction methods, the robustness and efficiency need to be further improved; the current stereo matching algorithm is inefficient, and it is difficult to match in the occlusion or low-texture regions accurately; virtual view synthesis method still defect in dealing with artifacts and holes, which affects the rendering quality.To solve those problems, this dissertation proposed a non-symmetrical DIBR code/decode solution, which contains efficient stereo matching, robust image correction, multi-viewpoint coding/decoding, high-quality virtual viewpoint rendering, and the method of 3D stereoscopic display.The main work and innovation of this dissertation are as follows:I) With the analysis of the key technologies, a code/decoder solution of DIBR free viewpoint video is proposed. In this solution, the sender will be in charge of the heavy computing tasks such as stereo matching, image correction and Multi-viewpoint Video coding, the receiver only needs to decode the videos and rendering the virtual viewpoint. This configuration reduces the display device requirements2)Proposed a color correction method based on SURF feature point matching to solve the occlusion, scaling, and rotation problems. which improves the multi-viewpoint coding compression efficiency, and improve the quality of virtual view.3) Aiming at the problem of low efficiency and mismatching caused by occlusion in global stereo matching, this dissertation proposed a novel method of stereo matching with high efficiency. Firstly, by initial local matching, occlusion and mismatched pixels can be checked out:then, the result of plane estimation is used to correct the initial match cost, thus pixels in occlusion region would be refined:at last, an improved hierarchical belief-propagation is used to estimate the minimal global energy to reduce the mismatching in low texture region and enhance the smoothness. Experimental results demonstrate the outstanding performance of the proposed method.4) For existing artifacts and holes in DIBR virtual viewpoint synthesis, this dissertation proposed a new rendering method on the basis of depth and image. Firstly, this dissertation presents a simultaneous rendering mechanism to generate the depth and image of virtual view at the same time, and then the ghosting is erased by the depth comparison; secondly, by means of depth-based hole filling and boundary treatment, the holes and boundary distortion is removed. Experimental results demonstrate that, by our method, the PSNR of virtual view improves 2 dB than MPEG-3DV/FTV reference method.