Research On Super-Resolution Reconstruction Of Light Field Image

Light field imaging is widely used in visual tasks such as digital refocusing,3D reconstruction,and salient object detection.Nonetheless,the resolution of 4D images captured by existing light field imaging equipment is generally low,which seriously restricts the further development and application of light field imaging technology.Light field image super-resolution(SR)reconstruction aims to reconstruct the corresponding high-resolution light field image from the given low-resolution light field image by image processing algorithm.This approach does not require improving the hardware performance of existing imaging devices,but instead uses computational methods to increase the spatial and angular resolution of light field images,making it a research hotspot in the field of computer vision.Recently,the SR methods for light field image based on deep learning have achieved high reconstruction performance.However,due to the high dimensionality of the light field image,the existing methods still face many challenging problems,such as the failure to fully exploit the complementary information between views,the inability to effectively model non-local attributes of 4D light field images,and difficulty in balancing the reconstruction quality of complex scenes such as occlusion and large disparity.Aiming at these problems,this dissertation has carried out in-depth research and exploration on light field image SR method.The specific research contents and contributions are as follows:(1)A spatial SR reconstruction method of light field image that fuses hierarchical features is proposed.To solve the problem that existing methods fail to fully exploit the complementary information between views,two kinds of light field reconstruction networks that fuses hierarchical features are proposed,namely hierarchical feature interaction network and enhanced separable convolutional network.The former effectively improves the network’s ability to represent the complementary information within and between views of the light field through feature interaction of spatial and angular dimensions.At the same time,a residual spatial-channel attention block is proposed to enable the network to recalibrate the feature map according to the information type and focus more on extracting the discriminative features.In view of the significant differences in information contained in spatial and angular dimensions of light field images,the latter adopts different feature extraction methods in these two dimensions to expand the receptive field and aggregate more angular information.In addition,an asynchronous multi-scale feature extraction module is designed,which effectively utilizes image features at different scales using asynchronous cascaded small convolutions and dense connections.The experimental results demonstrate that the proposed method can reconstruct light field images with more detailed information while maintaining the geometric structure of the light field.(2)A transformer-based light field angular SR reconstruction method is proposed.To address the problem that existing reconstruction methods based on convolutional neural networks have limited receptive fields and cannot effectively model non-local properties of 4D light field images,by introducing the transformer with global modeling ability into light field image angular SR,two modules are constructed,i.e.,angular transformer and spaital transformer,to capture the long-distance dependencies in the angular domain and spatial domain.At the same time,convolutional layers are added to the shallow and deep layers of the network to extract local features.The angular transformer converts light field angular features into a series of angular tokens and calculates self-attention in angular domain,so that each synthesized view can fuse complementary information from all input views based on attention weights.To reduce computational complexity,the spatial transformer divides the spatial features of light field into non-overlapping local windows and performs self-attention within each local window.Moreover,image and angular dense skip connections are utilized to enhance the information flow and feature reuse between different layers.Experimental results demonstrate that the proposed method can reconstruct dense and texture-clear light field images both on synthetic datasets with large disparities and real-world datasets with small disparities.(3)An attention-guided multi-dimension feature fusion light field image angular SR reconstruction method is proposed.To address the problem that existing methods have insufficient feature extraction capabilities and difficulty in balancing the reconstruction quality of complex scenes with occlusions and large disparities,a light field angular SR reconstruction network that integrates the multi-dimensional features and geometric structure priors of the light field is proposed.The network consists of three parts: a reconstruction subnetwork MFNet that fuses multi-dimension features,a reconstruction subnetwork GSPNet based on geometric structure priors,and a fusion subnetwork AFNet guided by attention mechanism.MFNet extracts intrinsic physical features of light fields from four dimensions including spatial,angular,epipolar plane,and pseudo-video sequences,and generates an initial reconstructed light field.It does not require depth estimation,thus avoiding the influence of factors such as occlusion and non-Lambertian surfaces.GSPNet explicitly models scene structure through a multi-scale encoder-decoder and projects input views to the target positions for synthesizing new views,which effectively preserves detailed information such as complex textures.AFNet adaptively integrates the reconstruction results of MFNet and GSPNet using attention fusion strategy to leverage their respective advantages.Experimental results demonstrate that the proposed method can not only reconstruct the dense light field images,but also effectively improve the reconstruction quality of occluded edges and complex texture areas.