Depth Estimation From Light Field EPI Based On Self-supervised Learning

Light field camera is a new type of multi-view imaging equipment,which can obtain the spatial and multi-view information of the scene through one shot.Scene depth can be estimated from light field data,which is widely used in computer vision fields such as 3D reconstruction.Now,the performance of light field depth estimation based on deep learning has been significantly improved,but with the increase of input costs and the deepening of the network,existing light field datasets are difficult to support training of large-scale networks.At the same time,the network after training for synthetic scenes does not perform well in real scenes.To solve above problems,we focus on light field depth estimation about refocusing and EPI.The main work is summarized as follows:(1)We summarize the current research status of depth estimation from light field and self-supervised learning,and describes the theory of light field imaging.The basic principle of depth estimation using sub-aperture images,EPIs and refocusing cues is introduced.(2)We design a data augmentation based on EPI refocusing.By analyzing the relationship between the disparity shift and EPI slope of the scene before and after refocusing,epipolar maps with different slopes and corresponding disparity values are obtained in the same scene,which provides more training samples for depth estimation network.The experimental results verify the relationship between the disparity shift and epipolar slope.(3)We combine the self-supervised learning to implement depth estimation from the EPI.According to the structure characteristics of EPIs,the EPI-based relation network is constructed by fusing the epipolar line relation features.And the self-supervised training of the network is implemented by estimating the disparity shift before and after the refocusing.Experimental results show that the method achieves high estimation accuracy in complex and multi-occlusion scenes,especially in real scenes.