Research On Image Super-resolution Based On Light Field Depth Estimation

Image is an important carrier of information,and its quality directly affects the ability of information expression.Therefore,image super-resolution technology has a wide range of applications in many fields by its powerful ability to restore high-quality images.Compared with the traditional two-dimensional imaging,the images collected by light field cameras satisfy the demand for high-dimensional information.However,due to the limitation of mainstream light field camera structure,the acquired light field image has a trade-off between spatial and angular resolution,and face the problem of insufficient image resolution.The depth cue play an important role in light field super-resolution reconstruction,but most of the existing work focuses on how to use the depth cue,and neglects the deep mining and precision improvement of it,which leads to the unsatisfactory effect of super-resolution reconstruction.In response to the above problems,the paper starts from the two perspectives,uses the multi-focusness characteristics of focal stack and spatial characteristics of multi-view images to predict high-quality depth maps,and studies the light field super-resolution based on predicted depth cues.The main research work and innovation are as follows:(1)To solve the detail loss caused by defocus blur,which was failed to consider in the existing methods,a method of light field depth estimation based on multi-focusness information is proposed.This method mainly includes two modules: the context reasoning unit and the attention-guided cross-modal fusion module.Among them,the context reasoning unit fully extracts the multi-focusness information of the focal stack and the structural information of the RGB image,so as to explore the internal spatial correlation comprehensively.The attention-guided cross-modal fusion module uses multi-level attention weights to gradually fuse internal spatial related information from the context reasoning unit,so as to realize the maximum contribution of multi-modal features to the depth prediction.Experiments have verified the effectiveness of the proposed algorithm.Compared with other methods,this method can effectively solve the loss of details caused by defocus blur,has a more robust performance against noise and repeated textures,and can provide beneficial effects for tasks such as super-resolution.(2)To solve the artifacts and distortions in the synthesized novel views,which were caused by failing to make full use of the spatial relationship between multi-view images in the existing methods,a method of light field super-resolution reconstruction based on depth cues is proposed.This method decomposes the angular super-resolution reconstruction into two subtasks: view synthesis and light field optimization.For the flexibility of super-resolution tasks in practical applications,the view synthesis network introduces an unsupervised depth estimation module.The module effectively learns and integrates rich feature representations through parallel connection and multi-scale fusion strategies to capture long-range spatial relationships for improving depth prediction.Based on the predicted depth,the view synthesis network reconstruct the initial light field by the pixel shift on the input sparse view.Based on the relationship between reconstructed views,the light field optimization network optimizes the initial light field to obtain the final result.Experiments have verified the effectiveness of the proposed algorithm.Compared with other methods,this method can significantly improve the depth prediction,and make full use of the geometric relationship between views to reconstruct a dense light field from a sparsely sampled light field with a larger baseline.