Research On Super-resolution Imaging Based On Compressive Sensing And Deep Learning

Super resolution imaging is a technique to compute high resolution images by using single or multiple frames of low resolution images obtained by the imaging system.After decades of research,such technique has been widely used in many research areas,such as remote sensing,photography and hyperspectral imaging systems.It is expected that the super-resolution methods can bring many benefits,such as reducing the dependence of high-cost and high-resolution detectors,reducing the data size of acquisition,storage and transmission,and improving the accuracy of advanced applications such as object recognition in computer vision by means of preprocessing.In the research field of super resolution,there are various algorithms and methods.Among them,compressive sensing and deep learning are two typical methods with excellent imaging performance,which respectively use the principle of signal sparsity and the learning ability of deep neural networks to achieve the task of super resolution.This paper focuses on the subject of super resolution imaging and makes some innovative researches on compressed sensing and deep learning.The main work is summarized as follows.1.This paper not only proposes an innovatively improved super resolution algorithm based on compressed sensing and deep learning to achieve better imaging performance,but also successfully combines the algorithm with the optical imaging system to obtain excellent results in remote sensing and hyperspectral imaging applications.Such a solution of integration of software and hardware combines the advantages of both and realizes higher academic and application value.It is the embodiment of the combination of the theory and application,which has some pioneering significance in the area of super-resolution imaging.2.Based on the compressive sensing theory and multi-frame super resolution imaging technology,an experimental verification device for a super resolution imaging system that can be applied to remote sensing detection is designed and established.The high resolution digital micro-mirror array(DMD)is used as the coded aperture in the middle image plane.Its function is to encode the spatial information of the target image,obtain multi-frame low-resolution image by multiple exposures,and then compute the high-resolution image of the target object by computation.The reconstruction algorithm in this paper is mainly based on the compressive sensing theory.Discrete cosine transform(DCT)is selected as the sparse basis for image signal transformation,and a new variant algorithm based on iterative reweighted least square(IRLS)is innovatively used as the reconstruction algorithm for the super-resolution imaging process.The algorithm can also reconstruct high-quality high-resolution images with a sampling rate as low as 25%.In the process of experiment,this article also combines the non-uniform correction algorithm with the compressive sensing algorithm,which solves the nonuniformity problem caused by the optical miss-registration between DMD and image sensor.The non-uniformity can be reduced to 10% of the original.Through the quantitative evaluation of peak signal-to-noise ratio(PSNR)in the simulation experiment and the qualitative evaluation of visual effect in the actual imaging experiment,it can be concluded that this method can achieve better imaging performance.3.On the premise of further optimizing and improving the performance of superresolution imaging,this article innovatively introduces the deep learning theory and combines it with the compressive sensing super-resolution algorithm to realize a new single-frame super-resolution algorithm with excellent performance.In this algorithm,convolutional neural network(CNN),which is widely applied to image processing field in deep learning,relies on its strong adaptive ability to solve the high frequency noise of fixed pattern caused by the collocation of sparse basis and reconstruction algorithm in the compressed sensing super resolution algorithm.We innovatively cascade the compressive sensing and deep learning process.In the first stage,the advantages of signal sparsity in compressed sensing are applied,and in the second stage,the advantages of adaptive learning network in deep learning is applied.The combination of the two algorithms can achieve better super resolution imaging performance,and it can surpass the same type of algorithms in terms of peak signal-to-noise ratio,structural similarity and visual effect.In this article,residual learning and batch normalization operations are added to the structure of convolutional neural network,which reduces the demand for the number of training samples and the required training time of the whole system.Finally,it is applied to the thermal infrared band super resolution imaging task,which proves the universality and robustness of the algorithm independent of spectral band.4.On the basis of super resolution remote sensing imaging system,this paper innovatively designs and establishes a hyperspectral super resolution imaging system using liquid crystal tunable filter(LCTF)as spectrometer.Based on the original remote sensing imaging system,the spectral division module was added.The super resolution algorithm that integrating compressed sensing and deep learning was applied at each band.This system can be used as a verification prototype of staring hyperspectral super resolution imaging system.It can reduce the difficulty of designing large array hyperspectral sensors by transferring the design difficulty to high-performance computing and making full use of the computing resources which have flourished in recent years.The spectral range of the system can reach 420nm~720nm,and the performance of super-resolution can reach up to 2 times of original spatial resolution.Different types of hyperspectral super-resolution imaging algorithms are compared with our method on the open source hyperspectral image dataset,by comparing the mean square error of the super-resolution results.The system has a controllable and dynamic sampling rate,and provides a method of data compression.The data size can be compressed up to 25% of the original.