Depth Estimation From Singlmonocular Imagesvia Deep Learning And User Interaction

Estimating depth from single monocular images is a fundamental and important problem in computer vision,which has found wide applications in various tasks such as scene understanding,3D modeling and pose estimation.With the recent development of deep learning,deep convolution neural network(DCNN)based approaches have gradually become the main stream of single image depth estimation and keep refreshing the precision records on standard RGB-D datasets.However,existing DCNN-based depth estimation approaches also have problems such as high requirement for training data,hard to post-process according to user requirements and difficult to define cost functions which reflects the underlying structure of a real depth map.This thesis addresses these problems by integrating new components into existing DCNN-based depth estimation.First,residual networks and the conditional random field(CRF)are combined into a novel framework.The residual network can obtain a larger receptive fields of the image,extract more complex and reasonable features,and the conditional random field has a good constraint on the edge of the image.Experimental results show that the proposed method can recover reliable depth maps of high precision even when trained with a small set of image-depth training set.Second,this thesis proposes an interactive CNN-based depth estimation framework which is capable of learning from both RGB-D dataset and human feedbacks to generate depth maps adapted to user need.A user inspects the errors in the predicted depth maps automatically generated by DCNN and interactively corrects them by giving feedback on error regions.The DCNF model is fine-tuned based on user feedbacks and new depth maps are generated by the updated model.This thesis also propose a user interface to simplify the task.Finally,in order to generate more realistic depth maps,this thesis proposes a depth estimation framework based on Generative Adversarial Networks(GANs).Cascaded with an depth estimation network,we build a determinative network which is trained with ground truth and estimated depth maps respectively as positive and negative samples,and hence learns high level dependencies among pixels of a real depth map.This information is then feedback to the depth estimation network through adversarial training which force the depth estimation network to generate depth maps which fulfills these dependencies.