Depth Super-Resolution And Virtual View Synthesis In 3D Stereoscopic Vision

By introducing the most advanced computer graphics(CG)and 3D techniques,the film Avatar makes huge commercial success and ushered in the era of 3D film.In 2010,the 41st World Exposition was first held in Shanghai,China.During the expo,3D techniques have been applied in both the hope earth exhibition and the forest carbon sink exhibition in China Pavilion.This is another successful case of 3D techniques in engineering applications.The immersive and vivid visual experience stimulates consumers' deep desire to high quality 3D stereoscopic vision.Depth,as an important geometric cue in 3D vision,is the root cause of 3D-perception.High quality depth acquisition has became a hot issue in recent years.Depth super-resolution,which is an indispensable technique in active depth acquisition,has been widely researched.On the other hand,the lack of 3D contents and the visual fatigue caused by long-time viewing have hindered the development of 3D industry greatly.Depth based virtual view synthesis is an effective way for generating 3D contents,besides,the resolving of visual fatigue also depends on the synthesis of high-quality virtual view.In view of the problems in depth acquisition and depth industrial application,techniques of depth super-solution and depth based virtual view synthesis are studied.The contributions are summarized as follows:Aiming at the artifacts of depth confusion,depth bleeding and depth missing in the up-sampled depth maps,a series of region-partition based depth super-solution models are de-veloped.By increasingly mining the internal relations between low-resolution(LR)depth map and high-resolution(HR)color image,depth super-resolution models including the multi-scale color image clustering based models,the color image segmentation based model and the color image fuzzy-classification based models are put forward successively.In the multi-scale color image clustering based depth super-resolution models,depth is used for judging the viability of the clustered classes,then only the valid classes will be up-sampled in each layer,thus the depth edges are preserved to a certain extent.In the color image seg-mentation based depth super-resolution model,depth is used in the adaptive segmentation of the color image,so that the segmentation result greatly maintains the boundary consistency between LR depth map and HR color image.In the color image fuzzy-classification based depth super-resolution models,the LR depth map is segmented according to its depth in the first place,then the segmentation result is applied in the supervised fuzzy-classification of the color image.The final classification result not only preserves the depth edges well,but also suppresses the redundancy color textures effectively.On the basis of machine learning theory,the internal relations between depth map and color image are greatly mined within the proposed region-partition based depth super-resolution models.Due to the co-guiding of depth map and the color image,problems like depth confusion,depth bleeding and depth missing are resolved fundamentally.What is more,these region-partition based depth super-solution models are applicable to both static filtering based system and dynamic optimization based system.High-quality and HR depth maps with clear boundaries can be generated via the proposed models.From the perspective of industrial application,both multi-view generation model and stereo-scopic view generation model are researched.To solve the problems of ghost contours and color discontinuities in multi-view generation model,the boundary-aware 3D warping strat-egy and the color-correction based image blending strategy are developed,respectively.The modified multi-view generation model removes the ghost contours and solves the color dis-continuity problem caused by distinct shooting environments effectively.Thus the quality of the synthesized virtual view is significantly improved.In addition,two depth aided hole fill-ing models are presented for filling the holes brought by the virtual view synthesis models,namely the depth aided hierarchical hole filling model and the depth aided dictionary based hole-filling model.In both of the two models,depth is utilized for dividing background from foreground firstly,then background is preferentially selected for filling the holes.