Research On Virtual View Synthesis Methods For Stereo Video

In recent years, stereo videos are entering the home and mass entertainment market gradually thanks to its strong sense of reality and outstanding interactivity. Consequently, increasing number of3D movies are released in the cinemas, occupying the commanding heights of the movie industry rapidly; Three-dimensional television (3DTV) is taking place of high definition television (HDTV) gradually, becoming the leader of the television area. Good user interactivity is the biggest feature of multi-view video and free viewpoint video, for instance, the viewers can select the viewpoint as they like. In the video applications of this kind, the video information of multiple views is essential. What’s more, the number of views is the key factor which will directly determine the video effects. However, limited network bandwidth and high cost of cameras impose restrictions on the number of views in reality. Thus, the virtual view synthesis method is paid more and more attention.Among the various existing methods of virtual view synthesis, depth-image-based rendering (DIBR) is the most typical and efficient one. It is implemented through3D image warping to receive the virtual image, utilizing the known color image and depth image of reference view. By virtue of lower complexity and better rendering quality, DIBR has become the research focus in the area of view synthesis. Recently, thanks to the endeavors of numerous researchers, there exists some excellent research achievement in DIBR. Nevertheless, quality of the synthetic view is still limited by some problems, such as visibility, resampling, disocclusion and ghosts. It is impossible for us to receive a virtual image of higher quality and better viewing experience unless the above problems are well handled.With regard to the common problems in DIBR, two improved methods which are based on the space domain and space-time domain respectively are proposed in this thesis. The two methods refer to a DIBR method based on inverse mapping and depth-aided image inpainting and a targeted space-time hole filling method based on bidirectional warping for DIBR.A novel DIBR method which receives the virtual image through inverse mapping and fills the holes with the improved image inpainting algorithm is proposed in the first method. The inverse mapping is employed instead of the traditional forward mapping, which addresses the problem of visibility and resampling skillfully. Moreover, a modified image inpainting method with depth information in consideration is carried out to ensure that the holes area can be filled by background information. The related modification includes the optimization of the priority calculation formula which can ensure an optimal inpainting order and the improvement of the searching principle of the best-matched patch which is the key step to raise the accuracy of hole filling.A more accurate and effective hole filling method combining the space and time domain is proposed in the second method. Based on the traditional hole filling method upon the space domain, the temporal information is referenced to obtain the virtual image of higher quality in this novel method. However, compared to the other space-time method, it chooses to perform the backward searching and objective location to realize a targeted background restoration instead of restoring the whole background scene, which can reduce the data sizes and the complexity largely. The restored objective area can be used to fill part of the holes in the virtual image. Then the remaining holes can be inpainted through the image inpainting method proposed in method1.Experimental results demonstrate that the virtual image obtained through the above two methods is satisfying in both subjective effect and objective evaluation.