Depth Image Super-Resolution Based On Deep-Learning

With the development of 3D vision technology,the acquisition and processing of depth images become one of the hot topics in computer vision community.Depth camera based on Time-of-Flight(TOF)has been widely applied in logistics,security monitoring,robot vision,etc.To solve the issues of noise and low resolution in the original depth images obtained by TOF cameras,we carried out the study on depth image super resolution based on deep learning.The main contents of this work are as follows:(1)We elaborate the background and research status of the topic,and point out the existing problems in the depth image super resolution task.Meanwhile,the characteristics and advantages of convolutional neural network are analyzed,the principle of TOF camera and the structure of Color-Depth camera platform are illustrated.Then we explain the calibration principle of camera and the process of stereo registration in details.(2)Focusing on the issue of edges blurring and artifact in depth image super resolution,a depth image super resolution method based on convolutional neural network is designed and implemented.In this method,the depth images are directly regarded as the initial inputs of our network.A end-to-end deep network structure is.constructed by combining the residual network structure with the sub-pixel convolution layer.Experimental results show that the network has a faster convergence speed while obtaining better reconstruction results.(3)A Color-Depth camera platform is designed and constructed.On this basis,a depth image super-resolution method based on color image guidance is implemented.The method introduces the high frequency information of the color image in same scene on the basis of the traditional single-channel network,and constructs a double-channel network framework for the depth image super resolution.Considering the problem of noise in original depth images and structural inconsistency interference in color images,the edge-guided filtering is implemented.Experimental results show that our method can well protect the edge structure of image and achieve a good de-noising effect.