Research On Depth Estimation Of Light Field Image Based On Deep Learning

Recently,due to the development of computational technology,light field camera is able to capture both the spatial and the angular properties of light rays in space.Some applications,e.g.3D modeling,scene reconstruction,depth of field expansion,which are difficult for traditional images,have a good development in light field image.However,on one hand,there is a clear trade-off between the accuracy and the speed in traditional light field depth estimation methods.On the other hand,the learning based methods,which only consider one directional epipolar geometry of light field images,estimate depth with low reliability.The occluded,textureless and nosiy regions also influence the scene information extraction.In order to deal with the occlusion,weak texture and noise problems,this thesis proposes a novel learning-based light field depth estimation framework with 3D Convolutional Neural Network and U-Shaped structure.Firstly,a multi-branch input network structure is proposed,which is designed to solve the problem of the learning-based methods which only consider one directional epipolar geometry of light field images and predict depth with low reliability.In the proposed method,the horizontal,vertical and two diagonal directions of light field images are concatenated and fed into the network to extract the depth features,so that the network can fully extract different epipolar plane features.The disparity map of the central view is directly output,in which the reliability of the depth estimation is improved.In addition,3D convolution layers are introduced in the multi-branch to better learn the slope of the line in Epipolar Plane Image,especially for the occluded areas.Experimental results show that the proposed method can effectively improve the the accuracy of depth estimation,especially in occluded regions.Secondly,a self-encoded and decoded U-Shaped light field image depth estimation network is proposed to deal with textureless and noisy regions.After cascading the inputs of each branch,the characteristics of the surrounding pixels are learned through the down-sampling process,and then the pinhole-connection is used in the up-sampling process to reduce the loss of down-sampling.Experimental results show that U-Shaped full convolutional neural network is able to accurately extract the texture information in textureless and noisy regions and improve the related depth estimation results.The model is trained using the public synthetical dataset and compared with other state-of-the-art methods on different test datasets.Experimental results show that theproposed method outperforms other method in terms of numerical and visual comparison,especially in occlusion,noise,weak texture and fine structure regions.