Study Of Light Field Depth Estimation Based On Deep Learning

Depth estimation,which aims to calculate the distance between the target and the camera,is a key step in the application of computer vision such as 3D reconstruction and automatic driving,and outputs depth map.Different from the imaging process of traditional camera,light field camera captures the brightness and angle information of incident light at the same time,and captures 4D light field with only one lens and one exposure.Based on the biplane representation method,the information of 4d light field can be represented as a set of sub-aperture image array with disparity.Light field depth estimation is a kind of depth estimation method that takes sub-aperture image array as input,and it is more robust than the traditional depth estimation methods on texture-less,occluded and noisy image areas.Convolutional neural networks(CNN)have strong abilities in deep representation and fitting,we formulate depth estimation as a classification problem which CNN is good at.We will put forward three convolution neural networks in this paper:EPN,SOA-EPN and EPI-Refocus-Net,theirs abilities in dealing with low texture,occlusion and noise improve orderly and gradually.EPN is a convolutional neural network which is based on EPI(epipolar plane image)analysis.It takes EPI images in both horizontal and vertical directions as input and uses siamese network to extract the deep features of these two EPI images.EPN fuses the deep features and then outputs a disparity map.EPN ranks 3rd on the 4D light field benchmark(April 2017).To further improve the accuracy of depth estimation,a scale and orientation aware EPN network,SOA-EPN,is proposed.On the basis of EPN,SOA-EPN takes multi-orientation EPI images(horizontal,vertical,45 degrees and 135 degrees)as input,and use some tricks such as adaptive scale selection network ASSN,shared weight network with four sub-networks,multi-layer cross-entropy loss to improve the accuracy of EPN in texture-less and occluded image areas.The mean error of SOA-EPN is 3.7 percentage lower than that of EPN,ranking 2nd on the 4D light field benchmark(January 2018).EPI-Refocus-Net is a convolutional neural network which integrates EPI cue and refocusing cue for depth estimation.It is based on SOA-EPN,and use some tricks such as using independent convolutions on height,width,channel dimensions,and introducing central sub-aperture image as a reference,to improve the accuracy of SOA-EPN in texture-less and noisy image areas.The mean error of EPI-Refocus-Net is 2 percentage lower than that of SOA-EPN,ranking 3rd on the 4D light field benchmark(December 2018).