Research Of Virtual View Rendering For Multiview Video

As the main way for people to get an access to the world, visual information has gained a rapid development in the past few decades. So far, Two-dimensional high-definition video has been widely used while the multiview video will become the next central issue. Multiview video present the users not only ordinary two-dimensional impression, but also depth information, which provides the users three-dimensional visual experience. Multiview video technology has wide application prospects in three-dimensional television, free viewpoint TV, video conferencing and virtual reality. However, multiview video has a huge amount of video data, which brings tremendous challenges to the video data compression, storage and transmission. A program to solve this problem is to take sparse camera array when capture videos, and render the virtual viewpoint after decoding, which can achieve higher coding efficiency.This thesis introduces the overall multiview video systems. It contains several modules named the signal acquisition, transmission, virtual viewpoint rendering, stereo display. Then, the thesis discusses the basic principles of depth based virtual viewpoint rendering, including pinhole camera model and the geometry relationship between a three-dimensional point and the corresponding pixel in the image plane.This paper mainly discusses a depth information based viewpoint rendering technique - 3D warping technique. This section first introduces the classic program of 3D warping, and analyzes the problems of the method, that is empty areas, cracks, occlusion areas and ghost profile artifacts. After that, the paper presents several programs for the problems and analyzes the advantages and disadvantages. Finally, the solution of this article, reliable and unreliable regions based rendering algorithm is provided. First of all, the reference viewpoints are divided into reliable part and unreliable part according to the position of the main reference viewpoint and the virtual viewpoint, and then the reliable part is warped to the virtual viewpoint according to the 3D warping method, and then the disocclusion part of the candidate virtual viewpoint is filled by reverse warping based on the virtual depth map synthesized before, the last step is to inpaint the virtual viewpoint based on the depth map to acquire a further improvement of the virtual viewpoint. Experiment results show that the proposed algorithm can solve the problems such as disocclusion regions, cracks, occlusion regions, and ghost contour well. The synthesized viewpoints not only have good visual impression, but also get relatively high objective indicators.