The High-resolution All-in-focus Image Fusion Technology Based On The Light Field Reconstruction

A sharp image with the information of a wide range of scenes is needed in military reconnaissance,medical detection,geological prospecting,and autonomous driving.It requires to obtain an image with the large depth of field and high-precision on optical imaging.Traditional cameras are inherently limited by the optical system,so the depth of field is also limited,which means than objects within the depth of field are focused and clear but objects beyond the depth of field are blurred.It makes the image sensor unable to clearly record all-in-focus information because of the information loss.All-in-focus imaging technology is the most common and effective method to achieve depth-of-field expansion.The most common all-in-focus imaging method is the multi-focus images fusion.This method can be mainly divided into two steps:the acquisition of the multi-focus images and all-in-focus fusion.In terms of acquiring multi-focus images,manual focusing methods have low accuracy and low efficiency in acquiring multi-focus images;the methods based on mechanical structures increases the complexity of the system because of the mechanical components,and the achievable depth-of-field extension is limited;the method based on specific optical components increases the cost and the adaptability of different scenes is poor,which is not conducive to wide application.In addition,the traditional multi-focus images acquisition method requires multiple exposures.This operation will cause inconsistent background information,which greatly affects the quality of all-focus fusion.In terms of all-focus fusion algorithms,it mainly includes methods based on spatial domain,transform domain,spatial domain and transform domain combination,and deep learning methods.Among them,transform domain-based methods such as wavelet transform fusion method have no translation invariance.The fusion error of this algorithm is large,and the information redundancy is large,too;the fusion method based on the space domain will have a large time loss,and the fusion quality of the pixels at the edge of the region will be poor;the algorithm combining the space domain and the transform domain increases the system complexity,and the image sensitivity with inconspicuous feature distribution is low;the fusion algorithm based on deep learning has higher hardware requirements and is prone to noise,which affects the quality of all-in-focus fusion.At present,there is still a lack of a method which can obtain high-quality all-in-focus images.In order to overcome the problems in the traditional all-in-focus image fusion algorithm,this paper uses the guided filter based all-in-focus fusion images algorithm to obtain all-in-focus fusion.The guided filter is a non-linear filter,which can retain the edge information and it h high speed and high quality all-in-focus images fusion.In order to overcome the problems in the process of multi-focus image acquisition,this paper adopts the light field imaging method to achieve multi-focus images acquisition under a single exposure,which ensures the consistency of the background information of the multi-focus images and the completeness of the input information of all-in-focus image fusion.This method also has the advantages of low cost,simple system structure and wide range of depth-of-field,and effectively realizes the collection of high-quality multi-focus images.Combining the guided filter based fusion algorithm with light field imaging,this paper proposes a guided filter based single-exposure light field all-in-focus image fusion technology,which can realize the acquisition of high-quality all-in-focus image with a large depth-of-field.The guided filter based single-exposure light-field all-in-focus image fusion technology uses light field imaging technology to collect multi-focus images,which is characterized by obtaining the intensity information and angle information of the scene light at the same time under a single exposure.This operation has a very high temporal resolution,but the spatial resolution of the image obtained by light field imaging is low because of the limit from the number of micro-lens units and the number of corresponding sensor pixels,so it is impossible to use the guide filter to obtain a high-resolution all-in-focus image.In order to solve this problem,this paper proposes a deep neural networks based high spatial-temporal resolution all-in-focus imaging technology,and designs a convolutional deep neural network which can be used for single image super resolution of light field images.The multi-focus images with low spatial resolution are up-sampled by this network,which effectively improves the resolution of the multi-focus images,and finally the all-in-focus image with high spatial-temporal resolution is obtained by the all-in-focus fusion algorithm based on the guided filter,and the high-resolution image with large depth-of-field of a scene is obtained.The guided filter based single-exposure all-in-focus fusion technology proposed in this paper solves the problems such as low precision,high complexity,inconsistent background information,and loss of high-frequency information,and realizes all-in-focus images under a single exposure.In order to improve the resolution of all-in-focus images,the deep neural networks based high temporal-spatial resolution all-in-focus imaging technology is further proposed,and convolutional deep neural networks are used to achieve high-resolution images acquirement.The up-sampling of light field images effectively improves the spatial resolution of all-in-focus images,which obtains a high-temporal-spatial-resolution all-in-focus image,and achieves high-resolution all-in-focus image acquisition within a large depth of field.It is of great significance in the practical application of the images acquisition and positioning such as military reconnaissance,medical detection,geological prospecting,and autonomous driving.