Light Field Depth Cues Based Depth Estimation And Geometry Structure Inference

Making a computer capturing 3D information like human eyes is a fundamental task in computer vision.The depth perception of human eyes involves jointly effect of multi depth cues,but traditional computer vision algorithm estimate depth from disparity cue on multi-view images.It is an ill-posed problem to reconstruct 3D structures from 2D images,so the depth cue of disparity suffers from unrobustness and ambiguity on pixel correspondence.An innovation in depth estimation comes from widely application of light field cameras.Light field could record the information of rays of a scene,capturing the intensity and angular samplings of rays,so it has advantages on depth cues extraction and fusion for depth estimation of scenes.The 3D reconstruction based on light field could be the key technique of light field theory and a promising issue of computer vision.The work in this dissertation is inspired from the principle of depth cue perception and fusion of human eye,focusing on depth cue extraction and fusion based on represen-tation model of Light fields.To handle the complex scene,it is studied on the dense depth estimation based on geometry inference in an energy minimization framework combined with scene prior,and the main contributions of the work are as following.(1)The traditional stereo matching is lack of robustness.A cost function fusion strategy based on feature matching under global optimization framework is proposed.The traditional disparity cues are extracted by evaluating matching cost functions,but solitary cost is susceptible to some difficulties.Combining cost functions with complementary properties could enhance the robustness and local confidence of the data term in global optimization.A visibility constraint is also proposed to handle with dense depth map estimation and local smoothness for multi-view cameras.It is validated the proposed algorithm on several real scenes captured by our camera array and Middleburry Stereo Datasets in the depth estimation experiments.(2)To model the scene by planes for geometry inference,It is proposed to detect a flat surface from original scenes based on EPI representation of light field.The geometry structure information of scenes are implicitly included in the light field data.A flat surface corresponds a fixed linear equation for all the EPIs.The proposed algorithm is based on EPI rectification and feature point extraction on EPI with a cross-verification strategy,and fit the plane function without any prior of depth estimation.The proposed algorithm effectively detects photo scenes from natural scenes from light fields in experiments,and the detection accuracy evaluated on EPFL light field datasets outperforms the comparing algorithm.The flat surface detection could be used for inclined plane refocusing and continuous depth estimation from light field.(3)For geometry inference on continuous surface,a depth estimation algorithm based on pixel-wise random plane assignment is proposed under the assumption that scenes are consisted of plane models.In the global optimization framework,the data cost is depth cue extracted from surface camera of light field,and the smooth term penalize the vari-ation of plane normals with 2nd order smoothness prior.Experiments on Middleburry Stereo Datasets and HCI datasets show promising results at depth estimation and geom-etry inference.The results is evaluated quantitively on HCI datasets and achieves the state-of-the-art accuracy.(4)A depth cue fusion method of light field under global optimization framework is proposed.For the problem of depth estimation on complex scenes with occlusion and non-Lambertian surface,a mask matching constraint of surface camera is proposed.A cost function on EPI depth cue and a fusion model combining EPI and surface camera is constructed to enhance the robustness of local depth cues.The algorithm is evaluated on HCI datasets and HCI 4D light field benchmark comparing with some the state-of-the-art algorithms and our method achieve the best performance at some evaluation metrics.The depth results on some complex scenes from Stanford Light Field Archive are also given.