Depth Image Super-resolution Reconstruction Based On Convolutional Neural Networks

As an expression of depth information,depth image is widely used in the computer vision field,such as three dimensional reconstruction,human gesture recognition,virtual reality and so on.Depth image is an image that describes the depth information of the scene.Compared with the traditional natural image,the quality of the depth image is not affected by the scene illumination and the reflection characteristics of objects in the scene.Depth image can accurately describe the depth information of the scene.However,due to the hardware condition limitations of depth image acquisition methods,the acquired depth image has very low spatial resolution and this depth image depth can not meet the needs of practical applications.Super-resolution algorithm to enhance the resolution of depth image has been widely studied by scholars at home and abroad.The critical factor of depth image super-resolution is to construct the mapping relationship between low resolution depth image and high resolution depth image.Because of its powerful nonlinear expression ability,convolutional neural networks have achieved great success in the field of natural image processing.At present,many researchers have applied convolutional neural networks to solve the super-resolution problem of depth image.However,due to few structural features of depth image,it can lead to error propagation dispersion.It can result in poor resolution of depth images.Recently,some scholars have made use of the similarity between the depth image and the color image in the same scene,and proposed a color image guided depth image reconstruction algorithm.For example,the based on filter method only utilizes the linear relationship between the depth image and the color image,only takes into account the information of the depth image and does not utilize the information of the training setdatas.Based on convolutional neural networks,our depth image super-resolution works are outlined as follows:1.In view of the problems existing in traditional convolution networks,we study and improve the structure of convolutional neural networks.This thesis constructs an end-to-end convolution neural network which use its nonlinear learning ability to learn the feature mapping low resolution depth image and high resolution depth image.We join the batch normalization layer in the network structure,which can fast convergence speed,have no neuron saturation problem,and can slow down the gradient dissipated phenomenon in the process of the back propagation.This convolution neural network model is used to superresolution depth image.Experimental results show that the proposed algorithm outperforms the existing advanced depth super-resolution methods in classical test datasets.2.According to shortcomings of the above based on convolutional neural network algorithm only using low resolution structural information of the depth image,without the using of color image information,this thesis proposes the based on convolutional neural network algorithm combined with the color image for depth image super-resolution.In order to use the color information of the image,we design a convolutional neural network filter based on three-dimensional structure.In this algorithm we use the two images as the input of the convolutional neural network,use three dimensional filter to learn the feature maps,which can learn nonlinear mapping relationship more accurately.And restructure the high resolution depth image from the learned mapping relationship.The simulation results show that,compared with the method of work and traditional methods,combined with the color image to restore better high resolution depth image,performance is better than other methods in the test data set of Middlebury.The experimental result prove the effectiveness of the proposed algorithm.