Depth Map Super-resolution Based On Convolutional Neural Network

Depth map acquisition technology is very popular in the field of computer vision.It plays an important role in 3D reconstruction,robot navigation,gesture recognition,movie games,virtual scene modeling and so on.Due to the limitation of the hardware device,the obtained depth map has a lower resolution and an inaccurate depth value.In order to solve this problem,many excellent depth map super-resolution(SR)methods have appeared.Depth map SR methods include traditional methods and methods based on convolutional neural networks.Convolutional neural network can automatically learn appropriate representation features,and has been widely used in the image SR.Based on this background,we use the convolutional neural network to realize depth map SR.The main research includes three aspects.First,we builds a model based on shallow convolutional neural networks.It realizes the feature extraction through the convolutional layer,and achieves the end-to-end mapping by deconvolution.The model solves the problem of high computational complexity and inaccurate feature extraction of traditional algorithms.Secondly,in order to further improve the reconstruction quality,we constructs a model based on two-channel convolutional neural networks.It includes two channels,the deep channel through feature extraction,nonlinear mapping,upsampling,and multi-scale to learns the detailed features The shallow channel is used to learn the rough features such as the boundary and angle of the depth map.Then combine the two-channel to finally achieve SR.Thirdly,we builds a multi-channel and multi-scale convolutional neural network to achieve SR task.It uses multi-channels to achieve fast extraction of image features,and uses multi-scale to complete image scale index information learning.In order to prove the validity of the model,we trained and tested the model on the color image and depth map datasets.The results show that the model can achieve good results on both types of image SR,it provides a possibility for the application scenarios which need to achieve color image and depth map SR simultaneously.