Hole Filling In2D Video To3D Conversion

With the development of display devices,3DTV is popular on the market and auto-stereoscopic TV is coming into our life. The3D display brings more realistic and shocking experience to us, but the contents for auto-stereoscopic are far from enough to customer satisfac-tion. The short of3D content is one limitation of3D technology generalization, so2D to3D or stereoscopic to multi-view conversion becomes the key technology for3D video generation. Due to the advantages of backward compatible, high quality and flexibility, the depth image based view generation becomes the important method for3D creation. Using the texture image and corresponding depth map we could rendering virtual views, which has adjustable parallax with the original. Original views plus their virtual views compose multi-view contents for the auto-stereo-scopic display device. The method of using existing2D video generate multi-view video for auto-stereoscopic display devices plays the urgent needs of the market. Generally, the depth image based rendering is divided into three steps: depth map acquisition and processing,3D warping, disocclusion handing. Due to objects of scene usually have different depth and occlusion, the object or the background which is occluded in the current view, may be exposed in the virtual view. Disocclusion areas are often defined as holes due to lack of effective information. The quality of these filled hole areas will seriously affect the audience’s visual experience. Because of the wide variety of complex scene of video, synthesis disocclusion area’s real content is very challenging. Handling of the disocclusion area is one of the most important parts of the DIBR. The spatial coherence and the temporal stability are two important factors of the virtual view, accomplish these factors could bring good visual experience to the audience. Simple algorithms to fill the hole areas, fast but often cause the texture dislocation, blur, edge defects and interflicker.This thesis, aims at the problem of filling disocclusion in the virtual view which is generated by DIBR, analyzes existing algorithms, tries to use image neighborhood and video interframe information to fill the hole, to improve the quality of the virtual viewpoint. This paper discusses the impact of the depth map quality with virtual view synthesis and employs fast image inpainting based method and the exemplar based disocclusion filling method, finally improves the quality of the virtual view. This paper’s work is mainly composed with the following aspects:1) preprocessing of depth map, focuses on the affect of bilateral filtering, joint bilateral filtering, and spatial-temporal filtering to generate a virtual viewpoint, explores ways to improve the quality of the virtual view by preprocessing depth map. 2) the edge interpolation method to fill the hole area always attains defects, utilize fast image inpainting technology to fill the area. As a neighborhood approximate method, the calculation is relatively simple, simple texture region can get satisfactory result and the repair procedure is not affected by the quality of depth map.3) For the hole areas of video, use the method of exemplar block of image, tries to use informa-tion from inter frame. The occlusion area of the current view may appear in other frames, so searches corresponding information to fill these holes and samples the inter-frames or neighborhoods to find the most similar image blocks with hole area. This thesis mainly analyzes the key issues of priority calculation and the calculation of the matching cost of exemplar based method. Utilizes frame coherence of video, searches the required information and matches the best candidate to fill the area, make sure spatial and temporal consistency of the virtual view to achieve synthesis of the virtual viewpoint with high quality.4) Based on these main algorithms, develops a presentation software of virtual view genera-tion based on MFC and OpenCV.