Depth Estimation From 4D Light Field Images

In recent years,with the development of photoelectric technology and devices,new imaging devices are emerging,among which optical field imaging has attracted much attention due to its unique imaging process and advantages.In order to overcome the inherent limitations of the traditional imaging system,optical field imaging combines the optical system with the signal processing algorithm,and collects the four-dimensional optical field information through the optical device,which is two degrees of freedom more than the traditional imaging.Therefore,richer image information can be obtained in the image reconstruction process.The optical field images collected by the optical field imaging equipment,namely the optical field camera,have abundant image information,from which we can extract the depth information of the scene,namely the depth map.As a common way to express 3d scene information,depth has been widely used.However,due to limitations such as baseline length and accuracy of pixel point matching between images from different perspectives,the range and accuracy of depth map obtained are limited to a certain extent,and there is a large error in depth map of some pixel points.To solve the problem of too narrow baseline between sub-aperture images Fourier phase shift theorem is used to calculate the sub-pixel displacement between sub-aperture images.After the rectification of the sub-aperture image,the spatial displacement is converted to the frequency domain,and the sub-pixel displacement of the sub-aperture image is calculated by Fourier transform and inverse Fourier transform.Compared with the method of traversing local patch,this scheme can effectively solve the problem of excessively narrow baseline between sub-aperture images and accurately calculate the displacement information of sub-aperture images from different perspectives.After the image position information of different perspectives can be accurately located,we will match the images of each perspective and the images of the central perspective respectively based on the global stereo matching algorithm.The energy function will be constructed by using the accuracy and smoothness of matching,and the initial depth map will be calculated.Because the energy function cannot fully constrain the matching,there is still the problem of inaccurate depth information of some pixels on the initial depth map.In this paper,a new optimization method is used to optimize the initial depth map by combining image restoration technology and depth map.By finding the corresponding relationship between the initial depth map and the optimized depth map,the loss function was established,and the SBI(split Bregman iteration)algorithm was used to iteratively solve the loss function,so as to effectively solve the problem of inaccurate depth information of some pixel points.In addition,we found that the edge part of the initial depth map always had more errors and the edge was not smooth.Therefore,after the edge information of the initial depth map was calculated by using edge detection in this paper,the confidence of edge pixels was reduced and used to construct the loss function.By optimizing the initial depth map in this way,the error problem of inaccurate pixels can be solved pertinently and the accuracy of depth map can be improved effectively.