Research On Depth Map Estimation Of Micro Lens Array Light Field Photography

Restore the real depth information of image and video is a classical problem in the field of computer vision,which plays an important role in robotics,target tracking,scene understanding and 3d reconstruction.Traditional digital camera only partially records a 2d plane projection of the real world so that it is difficult to reflect the 3d structure of the real scene,which hindered the further application of image and video.Lytro is a new type of light field camera based on micro lens array and records both the position and angle information of light at the same time.After an exposure a multiple perspectives,multiple focal plane images can be obtained.Light field camera is widely used in the field of image depth map estimation due to its easy operation,high stability and plasticity of pictures.Light field is the collection of light that from the arbitrary point and arbitrary direction in the space.Compared to traditional camera,a micro lens array is insert between the main lens and the sensor in the Lytro camera.The light that from the target point through the main lens and focus on a micro lens,and then spread out strength and direction component by the micro lens,which can record the entire light field and extend the traditional two-dimensional image to four-dimensional.In order to increase the rate of catching light,Lytro's micro lens elements are arranged in accordance with the hexagon,greatly increased the difficulty of sub-aperture images extraction and digital refocusing.So calibration and correction of the original light field image is very necessary.According to the above problem,this paper will focus on the Lytro light field image orthogonal transformation and depth map estimation.The main research content are as follows:(1)Thoroughly study the Lytro light field camera hardware and imaging mechanism.Directing at special arrangement of micro lens,combining with the knowledge of image processing,hexagon to orthogonal array calibration algorithm is proposed.This method can match micro lens with its cover sensor pixels and decode 4d light field data from 2d raw sensor data,Which can reduce the effect of intersymbol interference in the post-processing and verify this algorithm from two aspects of theory and experiment finally.(2)According to the transformation of the relationship between "free light field" and "pixel light field",a refocusing algorithm based on "pixel light field" is proposed to improve the efficiency of the algorithm and reduce the complexity of the operation.Take advantage of the feature of digital refocusing,a depth map estimation algorithm based on focus clues is proposed combined with the variable window clarity evaluation function.Finally,simulation experiments prove the validity and the correctness of the algorithm.(3)Take advantage of the parallax information between sub aperture images of Lytro camera,a depth map estimation algorithm based on the sub-pixel accuracy of stereo matching is proposed combined with guide filtering and image segmentation.Effectively to an end of the final depth map's visual effect and precision.And the algorithm is verified from two aspects of theory and experiment.(4)In order to eliminate the instability in algorithm(2)due to the window size and location,a depth map estimation algorithm based on focus and parallax clues fusion is proposed combined with algorithm(3).Effectively improves the stability and robustness of the algorithm.Finally,simulation experiments prove the effectiveness of the algorithm.