Depth Map Repair And Monocular Depth Estimation Based On Convolutional Neural Network

Depth estimation is one of the important directions in the field of computer vision,which enables computers to estimate the depth information of a scene through twodimensional images.At the same time,the convolutional neural network has powerful image feature extraction and function fitting ability,and the convolutional neural network has been applied to the depth estimation of the scene.However,there are still some problems in the current depth estimation method based on convolutional neural networks.On the one hand,the depth estimation problem based on convolutional neural networks is a data-driven model.Even if sufficient depth data can be obtained,most of the data cannot be directly used.For example,the depth map taken by Kinect is limited by hardware conditions in the original depth map.There will be a lot of voids that are ineffective in depth.On the other hand,although the application of deep neural networks to depth estimation has achieved quite good results,some of them have not been explained clearly.Some psychological studies have shown that there are some monocular depth cues,such as parallax,shadows,perspective,normal direction of the surface of the object,etc.These depth visual cues are the basis for humans to estimate the depth of the monocular.In response to the above problems,this is a series of studies around the monocular depth estimation based on convolutional neural networks.The specific research content of this paper is as follows:First,in view of the existence of voids in the original depth map captured by Kinect,this paper proposes a method for repairing the Kinect depth map based on convolutional neural network.The method first uses the depth map directly shielding the deep cavity area as the supervision.The information trains a monocular depth estimation model,and then fills the holes in the original depth map with the depth estimation result of the model,and then combines the filtering algorithm to make the depth map repair effect more natural.Second,in order to improve the ability of convolutional neural networks to estimate depth information,this paper introduces the visual cues of surface normal direction into the monocular depth estimation network.Based on the improvement of the monocular depth estimation model,this paper proposes a network that uses a single RGB picture to estimate the normal direction of the scene surface.Then a depth estimation network from coarse to fine cascade is designed,from which the clue of surface normal direction is explicitly introduced.Compared with the predecessors,this paper finds that the Kinect depth map repair method has obvious advantages in repairing large-area depth map voids,and this method is used for large-area voids,and the "depth" complements "depth" solution to determine the repair accuracy.It mainly depends on the accuracy of the depth estimation model itself.Here,if you use other depth estimation models with better effects,you will achieve better results.The cascading depth estimation network structure introduced in this paper also has a good performance in prediction accuracy.