Research On The Depth Propagation Algorithm For 2D-to-3D Conversion

Three dimension video(3DV)technology is under way to play a great role in areas of entertainment,medical service and education.3D video can provide an enhanced visual experience with depth perception beyond conventional 2D content.With the growth of 3D display devices,the increasing demand for 3D contents has aroused a significant challenge to the 3D video industry.One promising way to produce 3D contents is 2D-to-3D conversion.On the one hand,2D-to-3D converts 2D content into3 D.In preproduction phase,the production process is just like making 2D contents.There is few limits for shot scene of 2D video.On the other hand,2D-to-3D can make full use of existing classical 2D movies into 3D format,which can bring huge commercial value.Nowadays,commercial 2D-to-3D systems almost rely on labor cost,and the production period is too long.Semi-automatic 2D-to-3D uses manual input to assign depth for key-frames and estimate depth map of non-key-frames by depth propagation algorithm.Finally,3D videos are obtained with view synthesis.Semi-automatic technique can balance 3D video quality against manual cost.Depth propagation is one the most essential parts in 2D-to-3D system.Considering the drawbacks of state-of-the-art methods,this paper proposes two kinds of depth propagation algorithm.The main limitation of conventional motion-based depth propagation methods is that they are susceptible to inaccurate motion estimation and occlusion.To address these problems,we propose a two-stage structure-aware depth propagation method,which applies two different depth estimation schemes to the frames based on the temporal consistency.In the first stage,an initial depth map is generated by shifted bilateral filtering over the temporally consistent region.Due to inaccurate motion estimation and occlusion in temporally inconsistent region,motion-based propagation methods would introduce bad propagated depth errors.The second stage depth propagation estimates the depth of the temporally inconsistent region by solving an MAP approximation problem in the MRF model.In particular,an efficient priority belief propagation algorithm is developed,in which the priority of nodes to propagate messages depends on the structure saliency from tensor voting.MRF model can be efficiently solved by the proposed priority belief propagation algorithm,while preserving the structure in the depth map.This paper presents a sparse-to-dense depth estimation algorithm,which leverages the tensor voting at two different levels to propagate depth across frames.A 4D tensor voting is performed to remove outliers caused by inaccurate motion estimation,so that the depth can be propagated in accordance with the motion.In the second level,a high-dimensional tensor voting algorithm,incorporating spatial location,motion and color into the tensor representation,is devised to propagate the depth from the sparse points to the entire image domain.To extract local structures from the manifold that the points lies on,the tangent space spanned by the tangent vectors of the free parameters is utilized to establish the relation between the visual and motion input to the depth.By projecting the input feature into the tangent space,the relation between the location,motion,color and the depth can be established by voting process.Extensive experiments on public dataset validate the effectiveness of the proposed method in comparison with several state-of-the-art depth estimation approaches.