Model-guided Deep Hyperspectral Image Super-resolution

The Hyperspectral images contain a large amount of spectral information,and are far superior to RGB image data in identifying the physical characteristics of the research object.Therefore,hyperspectral images have many successful applications in the field of computer vision,such as object recognition and tracking.However,due to hardware’s limitations,it is difficult to directly image which is spatially high-resolution and spectrally high-resolution at the same time.There is a common solution using algorithms to fuse a pair of images with high spatial resolution and high spectral resolution respectively.The existing methods are mainly divided into two types: model-based super-resolution methods and deep learningbased super-resolution methods.Model-based methods often have limited performance and high time complexity;deep learning-based methods can be efficient but does not fully utilize domain knowledge,so there is still room for improvement.In addition,for simplicity,many research works use hypothetical degradation operators to simulate the degradation process of real images and generate training data.However,the degradation operators in actual scenes are often complex and unknown,so these methods would face the problem of inconsistency between the training data and the real data distribution,resulting in a significant performance degradation.Therefore,it is of great significance to study the blind super-resolution problem.In response to the above problems,this paper proposes a model-guided deep hyperspectral image super resolution method using both domain knowledge and deep image prior,so that the final super-resolution performance has been improved to a certain extent,and on this basis the blind super-resolution task was studied.The work and innovation of this article mainly include the following two points:1.Model-based super-resolution methods contain rich domain knowledge,such as linear observation models and hyperspectral image priors.However,they need to manually design image priors and iteratively solve the objective function during the solution process,which is very time-consuming.The methods based on deep learning is driven by big data,and they can capture the correlation between images of different spectrums,with high time efficiency and excellent performance,but it does not make full use of the domain knowledge of hyperspectral images.Therefore,this paper proposes a super-resolution method to expand the algorithm based on iterative optimization into the form of a deep neural network,which overcomes the problem that the model-based hyperspectral image super-resolution method relying on manually designed priors,and utilizes the powerful representation ability of neural networks to exploit image priors.Experiments show that the method proposed in this work is superior to other existing methods on both the CAVE dataset and Harvard dataset.2.For simplicity,it is generally assumed that the degradation operator in the observation model is known.But the inconsistent distribution of training and test data will cause serious performance degradation.Therefore,this work proposes to use a nonlinear convolutional neural network to implicitly model the degradation operator of hyperspectral images.Compared with the traditional linear model,the method proposed in this work has larger model capacity and better robustness.The experimental results on the synthetic dataset and the real spectrum dataset have proved the effectiveness of the blind super-resolution method proposed in this work,and outperforms several leading methods in terms of both implementation cost and visual quality.