Research On Depth Map Reconstruction Based On Joined Color/Depth Image Edge

In recent years,the emergence of RGB-D sensors,such as Time-of-Flight(ToF)and Microsoft Kinect camera,facilitate the real-time and low-cost depth capture,which is required in many applications,e.g..,3D reconstruction,view synthesize and 3D video.However,the depth maps captured by range sensors typically have low resolution,unknown regions and noise due to their intrinsic physical constraints.These deficiencies have impeded further applictions of similar range sensors.This paper intends to improve the resolution and quality of the depth data captured directly by using a registered high resolution color image.We investigate respectively the filter-based methods and optimization-based methods.The main works as follows:(1)We propose a texture edge-guided depth reconstruction method.We first analyze the edge similarity between depth maps and their corresponding color images,and then the incorrect region of the depth edge is extracted by using the recently structured learning approach.Finally,the incorrect regions are refined in an outside-inward refining order that is regularized by the detected true depth edges.Experimental results demonstrate that the proposed method provides sharp and clear edges for depth map,and depth edges are aligned with the texture edges,meanwhile,the 3D point clouds indicate that our proposed method yields a clear foreground and background.(2)We propose a depth map reconstruction approach based on Markov Random Field(MRF)model.We take into account the structure similarity between depth maps and their corresponding color images to improve the traditional MRF model.In the proposed model,the smoothness weighting is decomposed into two terms based on structure similarity and edge saliency.The results of upsampled depth map show that the proposed method can achieve better subjective and objective quality.The results of restorative depth map show that the proposed method can eliminate the influence of redundant texture information and preserve sharp depth edge.Therefore,filter-based methods can refine the local region,which is suitable for correcting edge regions of depth map.Compared with filter-based methods,global-based methods are more robust to upsample with high sampling rate.