Research On Deep Learning Based Image Super Resolution With Sparse Samples

As an important information carreier,image is the most important media for human beings to understand the world in the digital information era.Image resolution is a crucial evaluation index to assess the quality of an image.The image resolution indicates the information that the image contain.High-resolution images can recognize the small object and provide more tiny details on the observed objects.Thus,the demand for image resolution is always increasing.However,due to the limitations of imaging instruments and imaging environment in the imaging process,the acquired image have a relatively low resolution missing some details,which cannot meet the demands of daily life and industrial applications.Super-resolution reconstruction technology is a software-based post image processing technology,which can increase the resolution of images without updating hardware devices.Therefore,the cost of super-resolution reconstruction technology is relatively low and it has an important research significance for the applications in photography,video surveillance,remote sensing and medical diagnosis etcAt present,the research of super-resolution reconstruction algorithms are focused on two-dimensional(2D)image.From the viewpoint of algorithm principle,super-resolution algorithm can be can be classified into three types:interpolation-based,reconstruction-based and learning-based methods.Based on the spatial relationship of known pixels in the high-resolution grid,the values of the missing pixels can be estimated in the interpolation-based super-resolution method.The reconstruction-based algorithms consider the super-resolution reconstruction as an optimization problem,which introduce prior knowledge and have a better super-resolution performance on high-frequency regions.Compared with the first two types of algorithms,the learning-based super-resolution algorithm implicitly learns the prior knowledge of images by establishing a learning model,and has best super-resolution effects for different types of data.However,the performance of learning-based algorithms relies on the construction of a data sample library that requires a huge number of high-resolution samples to train the learning modelThis dissertation focuses on the high-dimensional image data such as 2D manifold and three-dimensional(3D)medical image.2D manifold data denotes the surface information of 3D object.The height of the surface can be considered as the grey value of the image if the 2D manifold is projected into a 2D space.Typically,this kind of data has a big dynamic range.3D medical image contains three dimensions data,the data complexity is significantly higher than that of 2D image.For these two types of data,it is difficult to collect enough samples to train a robust learning model.To address the above problem,this dissertation studies the learning-based super-resolution method with sparse samplesFirst,a 2D image super-resolution algorithm based on Neighbor Embedding(NE)and self-similarity is proposed.The algorithm establishes the mapping relationship between high and low-resolution image patches with fully exploiting the similarity redundancy in multi-scale and multi-angle levels.Meanwhile,an enhancement strategy based on local prior knowledge is designed to improve the super-resolution accuracy.The experiments performed on natural images and Magnetic Resonance(MR)images verify the feasibility and superiority of the algorithmSecondly,a DEM super-resolution algorithm based on gradient prior network and transfer learning is proposed.A novel algorithm that includes gradient super-resolution and transfer learning is proposed to apply the knowledge learned from natural image samples to the DEM super-resolution problem.On the other hand,considering DEM data is contiguous,the gradient range of DEM is much lower than its height.Therefore,the proposed algorithm implements super-resolution on the gradient domain.Finally,the final DEM data is reconstructed under the constraints on the gradient and height domains.A series of experiments show that the gradient-based method and transfer learning can still obtain a good super-resolution performance with limited DEM samples,which prove the validity and feasibility of the proposed frameworkFinally,a 3D super-resolution reconstruction algorithm based on multi-channel 2D convolutional neural network is proposed.For the three-dimensional super-resolution reconstruction problem,the purpose is to reduce the layer thickness of the three-dimensional data.The proposed method transforms the 3D reconstruction problem into multiple 2D image super-resolution problem.The multi-channel architecture is designed to simultaneously introduce the sequence images into the process of image super-resolution reconstruction,which ensures local consistency of 3D data.Moreover,the algorithm fuses different levels of super-resolution results to reconstruct the final isotropic data to ensure global consistency.In the experiment,the experiments show that the reconstruction quality of the proposed method is superior to other algorithm.Thus,this dissertation provides effective solution for high-dimensional image super resolution with sparse samples.