Research Of The Techniques Of Depth Estimation And Super-resolution Reconsturction Of Light Field

By collecting angular information of light,light field imaging achieves many novel applications such as post-capture refocusing and view-point switching,which are widely used in many fields such as security monitoring,virtual reality,and operation condition recognition.High-dimension and high-resolution light field is the basis for achieving good performance.However,light field imaging system is limited by the overall cost,acquisition equipment size and other limitations,resulting in the resolution of light field is low and the data is large,which restricts its further application.Therefore,it is important to improve resolutions of light field by using light field processing methods.Since light field are highly coupled,the resolution of light field can be effectively improved by decoupling the relationship of each dimension of the light field and using the complementary information among the each view of light field.At the same time,light field can be used to calculate the scene depth and generate the stereoscopic scene in the three-dimensional space.In addition,the depth information can provide guidance for the utilization of the complementary information of light field.In this dissertation,based on the two-parallel-planes imaging model,we study the light field processing and reconstruction techniques in both the spatial and angular dimensions,and the main research contents and works are as follows:(1)A light field depth estimation method based on multi-parallax integration is proposed to address the problem of matching the corresponding points of different views in strong noisy scenes.The method is based on the optical flow estimation method,and the initial estimated disparity is weighted according to the positions of different viewpoints in light field.To solve the difficulties of depth estimation in different regions of image,an outlier detection strategy is proposed for the screening of valid candidate disparities.Among them,for the occluded region,the detection of non-occluded viewpoints is added with the Lambertian assumption;for the texture-less region,the variance of angular patch is used for judgment;for the noise point,a noise elimination method based on the classification strategy is proposed.Finally,the effective candidate disparities are used as candidate values in Markov random field,and its optimal solution is performed using energy function.The comparative experimental results on real and synthetic datasets show that the proposed method can accurately estimate the light field depth and can better suppress the effect of noise on depth estimation,especially in strong noisy scenes.Moreover,the proposed method improves the accuracy of light field depth estimation in real scenes.(2)A light field angular reconstruction method based on correspondence field construction is proposed to address the problem that light field angular reconstruction is prone to produce blurring and artifacts in the reconstruction process of sparsely sampled light field.We capture correspondences among input views with a global receptive field.According to the inherent geometric structure of light field,the relationships of disparities and angular viewpoint positions are analyzed.A dense correspondence field construction method is proposed to effectively establish the correspondences between the input viewpoints and the viewpoints to be reconstructed.Based on the constructed dense correspondence field,a light field reconstruction process including view projection and feature projection is established.In addition,a projection loss is designed to constrain the construction process of the correlation field based on the correspondence between the input views.The comparative experimental results on real and synthetic datasets show that the proposed method can effectively perform light field angular reconstruction and accurately generate dense-sampled light field in large parallax scenes.(3)For the problem of incomplete utilization of information from different viewpoints in the light field spatial super-resolution,a light field spatial super-resolution method based on dense dual-attention network is proposed.Based on the analysis that the information in different views and different channels is of different importance,a view-wise attention module and a feature channel-wise attention module are proposed.Different weights are given to different views and different feature channels by these two types of modules.For the problem of information flow between different layers in the network,a dense connection is used.The proposed network takes light field images with low spatial resolutions as input.The network expands the perceptual fields through the multi-scale convolutional network,and extracts high frequency features of different viewpoints using the dense dual-attention network structure.Finally,it outputs high spatial resolution images of all views.The comparative experimental results on real and synthetic datasets show that the proposed method can recover high frequency texture details of light field images with a small computational cost.(4)A light field imaging platform is built,including a scanning light field camera and an camera array system with a high-precision scanning stage.The scenes include different depth-of-field and the same depth-of-field,corresponding to global different displacement and global consistent displacement.For the spatial super-resolution task,the standard light field datasets usually adopts the bicubic interpolation to simulate the degradation process,while the laboratory light field datasets has the real degradation process.So the feature extraction and matching of laboratory light field datasets are more challenging,which puts forward higher requirements on the adaptability and robustness of methods.Meanwhile,the laboratory light field datasets are also used for the light field depth estimation task,and the theoretical depth values are calculated by the light field imaging platform parameters and camera calibration parameters.Comparison with other methods verifies the robustness and adaptability of the proposed methods.