Research On Virtual View Synthesis Technologies For Three-Dimensional Television

With the emerging market in3D imaging products,3D video has become an active area of research in recent years.3D video is the key to provide more realistic and immersive perceptual experiences than the existing2D counterpart. Among many applications of3D video, Three-dimensional television (3DTV) is considered the main drive of the next TV revolution. Stereoscopic display is the current mainstream technology for3DTV, while auto-stereoscopic display is a more promising solution that requires more research endeavors to resolve the associated technical difficulties. Multi-view video (MVV) can meet the demand that many users watch TV simultaneously and it will get more attention and development in the future. In MVV, the number of view determines the watching quality. However, cameras’ cost restraints we cannot get sufficient views at the same time, and virtual view synthesis can achieve this goal.Depth-image-based rendering (DIBR) is an approach for target view images synthesis based on reference view image and associated depth information. It has attracted much attention due to its good coding efficiency,2D/3D selectivity, user interaction etc. Some previous work has been presented to render virtual view based on DIBR, however, there are still some problems of visibility, resampling and disocclusions in DIBR that need to be solved so as to acquire a high quality virtual view.This thesis focuses on the virtual view synthesis technology for MVV, which mainly proposes improved methods for the problems that exist in DIBR, and two different approaches are proposed, which are arbitrary view synthesis method based on DIBR and color image synthesis method for depth camera:(1) A novel method using two surrounding view images and associated depth images for improving the virtual view image quality is proposed. The key feature of this approach is that preprocessing the depth image is abandoned and a compromising way is taken into account. Compared with those complex hole-filling algorithms and time-consuming image inpainting techniques, some basic pixel-based operations are employed to fill the disocclusion areas in the virtual view image. A Horizontal, Vertical and Diagonal Extrapolation (HVDE) using depth information algorithm is proposed to fill the tiny cracks. The border of holes in the hole-flag map is distended so as to erase the ghosting errors. After complementation and blending, main virtual view image is obtained, then the image which is generated by filtering the depth image is treated as assistant image for filling small holes in the main image, and then final virtual view image is obtained.(2) A novel method using one reference view image and associated depth image to render virtual color image for depth camera is proposed. Pre-processing the depth image of reference view before3D image warping is disused for its disadvantages. In the process of warping, a priority formula is defined and the point which has the highest priority is the closest foreground point around the decimal pixel, and it is selected. Then Horizontal, Vertical and Diagonal Extrapolation (HVDE) using depth information algorithm is adopted to fill the tiny cracks. Next we make use of the depth image of the virtual view to decide whether the adjacent pixel of the edge of holes in the same row belongs to background or not, and then filling the holes using the background pixel value. At last the image is filtered to provide a natural appearance.Experimental results show that the two proposed methods can obtain satisfactory performance in both subjective quality and objective evaluation.