The Research Of Depth Estimation For Light Field Based On Convolutional Neural Network

With the development of computer vision,there are many new technology come into our life,like robot navigation,autopilot and medical surgical navigation.Most of these need depth information prediction technology,while it is the sticking points in computer vision.Most of the research has been working based on binocular vision.However,with the apearance and popularity of light field cameras,the estimation of depth information based for light field has gain peoples attention gradually.In recent years,the success of deep learning in the field of artificial intelligence has drawn so much attention.Many of these accomplishments are used by Convolutional Neural Network,which is a branch of deep learning.Convolutional Neural Network has achieved great success in many computer vision tasks such as image classification and semantic segmentation,which far more better than traditional algorithm.Bynow,there are few research on high dimensional light field depth information prediction with deep learning,how to effectively combine deep learning with the light field data is a problem needs to be solved.This paper proposed a convolutional neural network that can be applied to the light field data and generated extra light field data as training samples and finally predict the disparity map successfully.The main works are follows:(1)Because of the scarcity of light field data,this paper generated extra light field data by using open source software Blender.These data all contain disparity ground truth which can be used for training.In order to get better result with small amount of training data,this paper uses EPI Patch as sample to estimate disparity,and based on Canny edge detection filter out some bad quality sample,and finally uses oversample technology to balance the label distribution of training data.(2)This paper proposed an EPI Patch based convolutional neural network for depth estimation of light field.It used horizontal and vertical EPI Patch as input,and output the disparity probability distribution of each pixel.Since the result of CNN lacks global constraints and are unstable in occlusion or textureless regions,this paper use global regularization for CNN outputs,whitch makes up for the shortcomings of EPI Patch.Experimental results show that this depth estimation algorithm can get very well Bad Pix(0.07)score.It in the leading position compared with the publicly algorithm which take the same kind of input data.(3)Based on the above algorithm,this paper proposed a convolutional neural network with star shaped EPI Patch as input.This network receives 4 directional EPI Patch at same time,ie the 0 degree,45 degree,90 degree and 135 degree.And it used shared weighted network to optimize the speed of this algorithm.This paper further optimizes the network loss function design and adjusts the post-processing.Experiments show that the star network is superior than the cross network in most situation.