Research On Light Field Dense Reconstruction Technology

Densely sampled light field shows good advantages in applications such as depth estimation,refocusing and 3D display,but it is difficult and expensive to capture.Consumer portable light field cameras such as Lytro and Raytrix are easy to carry and easy to operate.However,due to the limitation of the main lens resolution,there is a trade-off between spatial resolution and angular resolution,and densely sampling cannot be performed simultaneously.In practical vision applications,the limitation of spatial resolution and angular resolution becomes the main bottleneck of handheld light field camera.And the problem of super resolution of light field has become a hot research area in computational photography.In recent years,on the basis of not changing the hardware equipment of light field acquisition,the super resolution of light field is reasonably and effectively realized in spatial and angular dimensions,which has become the focus of a large number of researchers.This paper deeply studied the dense reconstruction of the light field with sparsely sampled in angular dimension and wide field light field.The main works of this thesis are summarized as follows:(1)Dense reconstruction of light field based on separable convolutional neural networkIn order to solve the problem that the portable light field camera cannot sample intensively in spatial and angular dimensions at the same time,this thesis designs and implements a dense light field reconstruction algorithm based on adaptive separable-convolutional neural network.The main idea of the algorithm is to calculate the weight matrix of the input sub-aperture image on the target image through the convolutional neural network.The estimated weight matrix is then used to locally convolve with the input image to generate target sub-aperture images.Based on this idea,an adaptive separable convolutional neural network is designed,which uses the relationship between the input image and the target image to calculate the RGB value pixel by pixel to obtain high-quality target image.And a method of taking the position coordinates of the target sub-aperture image as the input of the network model is proposed.The algorithm uses sparsely sampled light field in the angular dimension to generate dense sub-aperture images reasonably and effectively.(2)Dense reconstruction of light field based on epi-polar plane interpolationDue to the limited field of view of the light field camera,capturing an image of light field usually cannot to meet the requirements in practical applications.It is time-consuming and expensive to capture the light field continuously in a dense panning light field camera,and it will generate a lot of redundant information.In this thesis,an light field dense reconstruction algorithm based on epi-polar plane image interpolation is designed and implemented.Because the slope of the corresponding line on the epi-polar plane images captured by the translation camera are the same,this thesis uses this property to match the corresponding line on the two epi-polar plane images.By interpolating the un-sampled epi-polar plane image,the dense reconstruction of the light field sampled from two viewpoints is realized.In this thesis,dense reconstruction is achieved for the light field sparsely sampled in the angular dimension and the large-field light field with sparse viewpoint sampling,and the reconstructed light field performs well in related q uality evaluation indexes.The work in this thesis will help to further play the advantages of densely sampled light field in a wide range of computer vision applications.