Study Of Depth Estimation Based On Depth Cues And Occlusion Detection For Light Filed Cameras

Light-field cameras capture not only 2D images,but also theangles of the incoming light.However,in practical application,the depth estimation results still face the challenge of accuracy and robustness due to the influence of light field camera resolution,illumination and occlusion.Therefore,in view of the problem of depth estimation for occlusion scene,two methods of depth estimation are proposed,which are based on geometric occlusion complementary model and epipolar plane image clues.Inspired by the traditional optical flow with occlusion detection,this paper focuses on the correlation analysis and the occlusion modeling for the sub-aperture array,and unifies them into a light-field flow framework.The main challenges faced are subpixel displacements and occlusion handling among the sub-aperture images.We build a light-field flow for joint depth estimation and occlusion detection,and develop a geometric occlusion complementary model.More specifically,we firstly estimate subpixel-accuracy optical flows from each two sub-aperture images by the phase shift theorem,then a forward-backward consistency checking is adopted to detect the occluded regions.According to the geometric complementary character of occlusion in a light-field image,an occlusion filling strategy is proposed to refine depth estimation in the occluded regions.Experimental results on the synthetic scenes and Lytro Illum camera data both demonstrate the effectiveness and robustness of our method which has excellent performance in handling occlusions.In order to improve the application of the above method to the light field angle information,a depth estimation method based on polar plane image clues is proposed in this paper,which is inspired by the line detection of polar plane images.The trajectory of the extraction point in the epipolar plane image in all angular directions is detected,so that more accurate depth estimation results can be obtained.At the same time,based on the feature of the trajectory segmentation in the epipolar plane image,a method for occlusion detection using the cost difference of the epipolar plane image is proposed.Experimental results show that the depth estimation based on polar plane image clues and occlusion information is very good.The results obtained by this method are better than those based on occlusion geometry complementary model.