Research On Depth Map Recovery Algorithms

Depth information play a fundamental role in many computer vision and computational photography applications,such as 3DTV,virtual reality,multi-view rendering,and autonomous navigation etc.However,due to the limitations of current active sensing technology,depth maps acquired by existing depth cameras,such as Time-of-Flight and Kinect,suffer from a variety of quality degradations,such as low resolution,noisy,and loss of depth.The low-quality depth information seriously affects its further application.Therefore,the recovery of depth maps is a very important issue.In this paper,we propose two depth image restoration algorithms based on the guidance of color image:1)Using the statistical characteristics of the depth image itself and the external constraints,an efficient depth image super-resolution framework is realized.Although there is a strong correlation between the depth image and the corresponding color image,there are obvious structural differences between them.Therefore,this paper proposes a depth super-resolution method combining the prior of the depth image itself and the prior of the external guidance image,namely the internal smoothness prior and the external gradient consistency constraint.Firstly,since the depth map is a piecewise smooth signal,we use graph smooth priors to recover it.In particular,we propose a new graph Laplacian regularizer,which achieves normalization,DC preservation and graph frequency interpretation simultaneously,and thus has more desirable filtering properties than existing ones.Secondly,we define a graph gradient operator to introduce gradient consistency constraint to enforce that the graph gradient of depth is close to the thresholded gradient of guidance,which further compensates for the problem of structural differences between depth and guidance images.Finally,the internal and external regularization is transformed into a unified optimization framework.Experimental results over widespread test images show that the proposed method outperforms existing color guided depth super-resolution methods.2)Combining the characteristics of local manifolds and non-local manifolds,a unified depth map repair framework is further proposed,which can handle various kinds of depth degradation.We propose this algorithm in a single low-dimensional manifold,providing low-dimensional parameterization of the local and non-local geometry of the depth map.On the one hand,a local manifold model is defined to favor local neighboring relationship of pixels in depth,according to which,manifold regularization is introduced to promote smoothing along the manifold structure.On the other hand,a patch-based non-local manifold model is defined to construct highly data-adaptive orthogonal basis,then a manifold thresholding operator is further defined in the 3D adaptive orthogonal spectral bases.The 3D manifold thresholding guarantees the strict sparsity of the signal decomposition on the manifold basis,to retain only low graph frequencies for depth maps recovery.Finally,we elegantly cast the adaptive manifold regularization and thresholding jointly to regularize the inverse problem of depth maps recovery.Experiments results demonstrate that under three typical cases of depth degradation,our method achieves superior performance compared with the state-of-the-art works with respect to both objective and subjective quality evaluations.