| Multi-view video has become the next generation 3DTV technology at 21st century. Compared with traditional 2D video, multi-view video can give the viewers 3D and immersive effect since it contained more video and depth information of the scene. Besides, its interactive function can provide different 3D effect from various viewing angles. Yet the capturing, transmission, processing and rendering of multi-view video as well as the huge data introduced by multiple video points are the bottleneck problem in the development of multi-view video technologies.3D system based on multi-view video involves multi-view data capturing, multi-view video coding and rendering virtual views at decoders. Multi-view data capturing is to acquire multi-view data which meet the requirement of system, multi-view video coding is for transmission and storage of multi-view data, rendering is to interactive display the 3D scene after reconstruction. The research in this paper is performed according to the flow chart system above, and 3D experiment platform based on multi-view video was proposed at the final stage.For the relation between multi-view data capturing and rendering, the constrain relation between factors of capturing system (such as spacing between each camera, camera's stance, number of camera arrays and quality of rendered virtual views was less considered when domestic and foreign research units were building up multi-view video systems. Most methods acquire high quality virtual view by increasing the number of camera arrays. Although the more the camera number, the denser the viewing zone, which will lead to a wider field of view angle and more realistic 3D effect, the huge multi-view data to be transferred will result in a higher demand for multi-view coding efficiency, especially the decrease of resolution is obvious for available multi-view display. Thereby focusing the problems above, we analyzed the multi-view video data based on camera models. Given that the multi-view data can be considered as a collection of ray data in a 3D space, the multi-view images can be viewed as a collection of ray which are either emitted from some light source or reflected by some object surface. So the 7D plenoptic function can be reparameterized to 6D. This 6D function is named as the surface plenoptic function (SPF). Based on SPF, factors including scene geometry, texture on the scene surface, reflection property of the scene surface, depth range of the scene, capturing and rendering cameras'resolution, focal length of cameras, arrangement of cameras and spacing between each camera, were considered comprehensively, and then by adopting signal sampling theory, we derived an optimal solution to reconstruct a virtual view. By analyzing the EPI of the scene, we obtained the relation between multi-view data capturing and quality of rendered view. According to the analysis above, a proper number of image samples were derived, a reconstruction filter were designed to interpolate enough sampled images and then rendered virtual views as required using IBR method for the camera model. Experimental results for both modeled scene and real scene show that proposed method can generate virtual views of good quality. Compared with traditional signal sampling theory, the number of cameras deduced and spacing between cameras are suitable for application. According to the relation between the number of capturing cameras, spacing between cameras and specific reconstruction method, the proposed method can only about 20% sample images are needed for rendering a virtual view, while the rendering quality remains higher.After obtaining optimal solution between number of camera arrays and quality of rendered virtual views, multi-view video coding is another key technology to be further investigated. Compared with traditional 2D video, multi-view video data contain video data from all cameras, so the data is huge. Since multi-view video is a combination of several video data for a same scene, there must be a high degree of correlation between multiple views. Therefore it is more important to reduce the spatial redundancy except for only reducing temporal redundancy like 2D video compression. According to the analysis above, we propose a multi-view video coding scheme based on novel"resultant vector"estimation. The loop constraint between disparity vectors and motion vectors in traditional stereo video was fully explored and extended to multi-view video coding unit. Meantime focusing on the parallel camera geometry model, coding efficiency was promoted by utilizing the linear relationship of adjacent views. Experimental results show that our scheme's performance is superior to conventional methods such as Simulcast and JMVM. Coding efficiency is improved about 0.2~0.5 dB.Based on the research above, after setting up proper camera capturing system and proposing efficient multi-view coding method, how to design a virtual view rendering method and improve its quality remains as another important key technology in multi-view video system. For acquiring a virtual view of better quality, methods available at present need dense camera array to capture a huge number of multi-view data, which will lead to a heavier burden of multi-view video coding as stated above. While if choosing sparse camera settings instead of dense camera array, using interpolation methods or up sampling calculation for rendering a virtual view will result in severe distortions in synthesized images and great decrease of image quality. So a novel virtual view synthesis algorithm is proposed based on corrected region based surface mapping and image fusion method to reduce perceptual errors and improve quality of virtual views. Since depth information is the key to depth Image based rendering, a plane sweep based depth correction method was presented when obtain depth information. The experiment results imply that proposed method improves the objective and subjective performance simultaneously, compared with previous dominant multiple image view synthesis approaches.
|
PDF Full Text Request