Depth Image Super-Resolution Based On Color Constraint Information

Consumer depth cameras(such as ToF and Kinect)can capture the depth information of dynamic scenes,but the acquired depth image has low resolution,contains a lot of noise and even the loss of depth values in some areas,which limits further application.In this thesis,we focus on low resolution,depth holes and noise problems,the related issues of the depth image super-resolution reconstruction are studied from the perspective of color prior constraint information.The main work of this thesis is as follows:(1)We describe the background of the depth images super-resolution reconstruction,analyze the significance of the topic,introduce the research status of the topic,and analyze the difficulties.At the same time,we introduce the degradation model of depth images,the calibration and registration principles of the RGB camera and the depth camera,and the basic filtering model of the depth image super-resolution.(2)For the depth image super-resolution,we design and implement a robust imageguided depth super-resolution algorithm.Firstly,the proposed RGB-D structure similarity measure is used to select the optimal neighborhood patch.In the selected image patch,the pixel depth value is estimated by the proposed multilateral guidance based on the oriented nonlocal means weight.Further,the multilateral guidance parameters are adaptively adjusted according to the smoothness of different depth regions.The experimental results show that the proposed algorithm can effectively suppress texture copying artifacts on synthetic datasets and real datasets,while preserving the discontinuity of depth edges and achieving robust depth image super-resolution.(3)In order to solve the depth hole problem in the raw depth image acquired by the depth camera,we design and implement a hole restoration algorithm based on sample patch matching.The algorithm utilizes color and depth images to extract probability contour information.Firstly,the sample patch is coarsely matched to suppress the local optimal phenomenon that may occur in the sample patch matching.Further,in the fine matching,the algorithm adds structure similarity measure and distance constraint information,and proposes a sample patch size adaptive matching scheme to effectively improve the matching accuracy.The experimental results show that our algorithm can effectively restore the hole regions and provide a basis for super-resolution.