Research On Key Techniques For View Synthesis In Stereo Video System

As 3D movie viewing has become mainstream and the Virtual Reality(VR)has developed,audiences show much more interests in the visual enjoyment.3D videos and images are usually stored in stereoscopic format.The image format includes two projections of the same scene,one of which can be seen from the view's left eye and the other from the viewer's right eye,thus providing the viewer with the experience of seeing the scene in three dimensions.With this 3D trend,the demand of transferring ordinary 2D video to 3D stereo video is growing rapidly and widly used in multimedia area of computer.Stereo video generation aims to generate the other virtual view of the scene and applys special displays to ensure a user to see different views from each eye.Depth Image Based Rendering(DIBR)is a mainstream technique for virtual view generation.A typical 2D-to-3D conversin process consists of two steps: depth estimation for a given 2D image and depth-based-rendering of a new image in order to form a stereopair.Inspired by this,we propose two methods for 3D stereo video generation in this thesis.One technique pipeline is first estimating depth from a monocular canera,then obtaining the virtual view through DIBR.The other one is resorted to deep learning.We propose a new convolutional neural networks(CNN)architecture,generating virtual view through an end-to-end solution.We come up with different techniques for conventional view synthesis.We first get the depth information from monocular image,and then propose a novel depth image super resolution method with cascade random forests in order to get high quality depth image.Then,a saliency based disparity nonlinear transformation and a motion information based temporal RGBD disparity optimazition are proposed.We propose a multiscale image hole inpainting method based on coherency sensitive hashing(CSH).The end-to-end CNN architecture StereoFCN is proposed based on fully convolutional networks,which can obtain a pixel-level image synthesis.We train our model end-to-end on ground-truth stereoframe pairs with the objective of directly predicting on view from the other.Finally,experiments show the effectiveness and superiority of our method with a comformatable 3D effect.