Research On Spatial Super-resolution Based On Light Field Images

Compared with traditional cameras,light field cameras can capture the scene information from multi-views and are widely used in scene reconstruction,post-capture refocusing,depth estimation and de-occlusion.However,due to the limitation of light field cameras,the spatial resolution of images captured by a light field camera is generally lower than those captured by conventional cameras,which affects the various applications of light field cameras.And the light field cameras represented by the array cameras can partially solve this limitation by increasing the resolution of sub-aperture image,but since the far detected distance,weak radiation intensity and aliased projection imaging in the optical detection tasks dominated by space targets,the limited imaging resolution still limits the subsequent high-level understanding tasks.In order to solve the problems mentioned above,this thesis presents an extensive research on spatial super-resolution of light filed images,and array camera images with near space target.The main research contents are as follows:In terms of the complex four-dimensional structure of light field images,we propose a simple but effective Transformer-based method for light filed images super-resolution.In our Method,the angular Transformer can capture the high correlation features of light field images in the angular dimensions,and the spatial Transformer can model the longrange dependencies in the spatial dimensions.We validate the effectiveness of our angular and spatial Transformers through extensive ablation studies,and compare our method to recent state-of-the-art methods on five public light field datasets.Experiments show that our proposed method can make full use of the rich angular information and spatial information of light field images,and achieve the angular modeling with spatial awareness and the spatial long-range dependencies modeling within each sub-aperture image.In terms of the limited resolution imaging of space targets on the array cameras,we study the geometric relationship between space targets and array cameras.As the sparsity of space targets imaging on the image plane,we propose a space targets super-resolution method of array camera images based on sparse reconstruction.In our method,the sparse signals with spatial targets can be reconstructed and super-resoloved by using the transfer constraints between views of array cameras and the point spread function estimation of space targets imaging,Simulation experiments on array camera images show that our proposed method can effectively achieve the super-resolution of space targets,and realize the estimation of the number and location of space targets.In terms of the near space adjacent taregets,which cannot be effectively superresolved by most hand-designed prior methods,we propose a near space adjacent targets super-resolution method of array camera images based on convolutional neural networks.Simulation experiments on array camera images show that,by using our proposed feature extractors,the deep feature of near space adjacent targets can be captured efficiently,and the near space adjacent targets can be super-resolved effectively.And we design a mixed-angular-resolution training strategy,to make our method achieve consistent superior super-resolution performance on array camera images with different angular resolutions.