3D Reconstruction And Display Of Multi-view Video

With the development of auto-stereoscopic display and augmented reality, the way people perceive the world has changed tremendously. Compared with two dimensional images and videos, multi-view content contains color information from different positions and corresponding depth information, which could provide users with immersive visual feelings.The researches and applications of 3D content are developing rapidly. However,3D production technology still faces many problems, including the lack of 3D content and the diversity of viewing conditions.The "2D video plus depth" format has the advantages of low transmission cost and terminal flexibility, becoming an important research direction. Based on the format, this paper mainly carries out research on the field of multi-view synthesis method and planar 3D display technology. Firstly, a new view synthesis method with superpixel hole filling is proposed. Secondly, the disocclusion problem in multi-view video is studied by utilizing the spatial and temporal information. Lastly, the stereo parameters adjustment for planar 3D displays is presented to achieve natural and comfortable viewing experience. The main achievements of this paper include:(1) Based on the superpixel segmentation and hole filling algorithm, a new virtual view synthesis method is proposed to solve some known issues, including ghosting, visibility, resampling, disocclusion and so on. A superpixel segmentation is applied on the original color image to get the initial sample space, and an oriented hole filling method is proposed to fill the disocclued regions. Experimental results demonstrate that the quality of the synthesized virtual view images are improved with the proposed method.(2) A spatial and temporal disocclusion information based hole filling method is proposed for multi-view video system. The spatial and temporal disocclusion information in the video sequence is accumulated for filling those disocclued regions. The hole filling results can achieve spatial and temporal consistency. Besides, a pre-processing procedure is used to deal with different camera motions, which extends the application range.(3) For planar 3D displays, a novel parameter adjustment method for reducing visual discomforts and geometric distortions is proposed by combining the geometry of stereoscopic capturing system with the comfortable viewing mles for the human vision system. An identically sized representation between the virtual space and the display space is established. The capturing and displaying parameters are adjusted according to the comfortable viewing zone and personal preference. Experimental results demonstrate that comfortable stereo perception can be achieved.