Research On Depth Retrieval For Lenslet Based Light Field Imaging

Micro-lens based light field imaging is a newly developed technology for threedimensional imaging,by combining computations and optics.Light field cameras of this kind have concise optical systems and can capture 4D light fields in a single exposure.In addition,it enables plentiful unique post processing that cannot be done by traditional cameras.Therefore,the micro-lens based light field cameras can be widely used in 3d reconstruction and measurement,3d detection and recognition,virtual and augmented reality.For many light field applications,depth estimation is regarded as the most important intermediate results for deploying high-level tasks.Depth estimation for micro-lens based light field cameras has been developed rapidly,but there are still many problems to be solved for practical usage,such as sub-pixel matching and occlusion handling.In this thesis,we focus on the depth estimation for micro-lens light field cameras and conduct systematic researches on theories and experiments.Firstly,existed light field theories are discussed in the thesis,especially the light field disparity and epipolar image theories.By wavefront analysis for micro-lens light field cameras,we introduced new understanding for light field imaging,and discussed the relationship with the previous theories.In conclusion,the micro-lens light field camera can be considered as a wavefront sensing device calibrated by spherical wave and suitable for extended targets.The depth estimation ability comes from the optical encoding and digital decoding of the wavefront originated from the objects.Based on the analysis,we formulate the relationship between depth and wavefront slopes.Combining light field dynamic reparameterization,we proposed to measure the curvature of wavefront through angular patches,and finally realized high-accuracy subpixel depth estimation.Secondly,in order to handle the losing of photo-consistency property when occlusion happens,by modeling occlusions in the proposed framework,we pointed out the similarity between the occlusion boarder and the edge in the angular patch at the correct depth.Therefore,we proposed an accurate depth estimation method by using multi-orientation angular coherence.Experiments in public datasets demonstrated the superior performance,in which the mean square error is decreased by 3 square pixels and the ratio of points with error more than 0.07 pixels is reduced by 10%.In addition,the accuracy,fine structure maintenance and generalization ability is also validated in real world experiments using different kind of micro-lens light field cameras.Conclusion can be drawn that the accuracy is guaranteed in non-occluded areas and improved around occlusion regions.To handle occlusions with complex shapes,based on the previous method,we proposed a new depth estimation method using adaptive region-based angular coherence.The method adaptively selects the sub-regions in the angular patches by Gaussian weighting.Experiments show that the mean square error is reduced by 4 square pixels and the ratio of points with error more than 0.07 pixels is reduced 11%.The improvement comes from the better handling of complex occlusions and the fine structure is maintained in the depth maps.Finally,to test the occlusion handling ability,experiments based on standard objects were conducted and the ability to handle different occlusions was also tested,proving the high performance of the proposed methods.All in all,in this thesis,the depth estimation from a micro-lens light field camera is achieved.Concretely,the imaging laws for the wavefront sensing in the light field camera is derived and two algorithms to handle occlusion are proposed.Based on all of these works,the accuracy and the fine structure of the depth maps obtained from microlens based light field camera are improved,with further advantages to extend the performance in high-level light field applications.