Content Aware Based View Synthesis

The rapid development of 3D display technology is in the process of changing the standard video format.Compared with the conventional binocular stereo video,multi view video can provide a vivid 3D experience with the advantage of rich content.However,this realistic 3D experience is at the price of enormous data amount,which has brought great pressure to image acquisition equipment,storage device and transfer pipeline.View synthesis can be an efficient method to solve these problems.It can utilize the existing binocular stereo video as inputs.By combining the computer graphics and computer vision theory,the mapping relationship between video content and scene location can be established and the unknown video content can be rendered.In this thesis,content aware based view synthesis focuses on image feature information.Taking the saliency map as guidance,a transform function is calculated between views.The main contributions are in the following three folds:(1)Starting from the traditional feature based view mapping scheme,SOBEL edge points are extracted and described as SIFT descriptors to make the homography more robust.These two features work together to fulfill the task of feature matching and matrix calculating.In addition,to alleviate the black region on the synthesis image border owing to the parallax difference of adjacent views,an image blending method based on correlation matching is proposed.Experimental results have demonstrated the effectiveness of feature expansion and image blending.(2)A regular mesh based view warping method is proposed.Under the framework of image domain warping,in order to make full use of the image feature information,moving least square is utilized to control mesh deformation and then get the initial warping position of the grid vertices.Furthermore,an grid line constraint is added to energy equation to restrain the line bending.Experimental results have shown that the final view synthesis results can satisfy the human visual comfort.(3)Based on the theory of regular mesh warping mentioned above,considering the image saliency and region similarity,an adaptive meshing project is proposed.The current region is divided if it is remarkable and dissimilar.In the meshing process,a top-down quad-tree based segmentation approach is adopted.Specific traversal method is tailored to accommodate with the adaptive meshes.Experimental results have shown that the reduction of mesh quantity based on adaptive meshing reduces the computational complexity at the expense of slight decrease in synthesis quality.