Research On Hyperspectral Image Fusion Based On Matrix Decomposition And Deep Learning

High-resolution hyperspectral images possess sufficient spatial and spectral information,which are convenient for object detection,recognition and tracking,so they are widely used in military reconnaissance,agricultural production,disaster prediction and other fields.However,limited by the current imaging equipment,we can only obtain low-resolution hyperspectral images and high-resolution multispectral images.Therefore,how to use these two known images to reconstruct the desired image is the research content of hyperspectral image fusion.An important principle of hyperspectral image fusion is that the target image shares the same spectral information as the low-resolution image and the same spatial information as the high-resolution image.Following this principle,based on matrix decomposition and deep learning,three fusion approaches are proposed and verified on simulated and real datasets.The main work of this article is summarized as follows:Hyperspectral image can be decomposed into a spectral basis matrix and a coefficient matrix.By estimating these two matrices,the desired image can be reconstructed.Hence,this article proposes a fusion method based on matrix decomposition and super-resolution priors.Combining two observation models and a priori super-resolution image(an interpolated image by Bicubic algorithm),we construct two optimizing models to estimate the coefficient matrix and the spectral basis matrix,respectively.Since the models contain unknown spectral and spatial degradation functions,we design a self-adaptive learning network to estimate them.The experimental results illustrate that the proposed method has achieved promising performance in both simulated and real datasets.Besides,this method can also be used as a general tool to help other methods improve the performance.Considering the effect of deep priors,this article further proposes a fusion method based on a deep super-resolution prior.Specifically,we design an end-to-end deep convolutional neural network(CNN)to learn the mapping between the two known images and the desired image.The network considers the combination of local and non-local features,the fusion of features in different levels and multi-scale analysis.Distinguished from natural images,we tailor a special loss function for hyperspectral images,which covers three aspects: space,spectrum and structure.In order to further improve the performance,we return the predicted result as a prior to the general tool proposed by us to obtain the final fusion result.The experimental results demonstrate that the proposed method outperforms other compared methods,which has achieved better quantitative performance and visual effect.In order to bridge the traditional fusion models and the deep learning organically,this article also proposes a model-driven deep fusion approach.To estimate the coefficient matrix,we combine the fixed spectral basis matrix,two observation models and the sparse prior to construct an optimizing model.This model can be solved by alternating direction multiplier method(ADMM).In order to adaptively learn the parameters involved in the model,we unfold the iterative solution of the model into a deep CNN where the number of iterations corresponds to the number of network stages.Besides,considering that the desired image cannot be fully represented by the spectral basis matrix,we construct an information supplementing model to supplement the representation error.The experimental results illustrate that the method achieves excellent quantitative performance and visual effect on all datasets,which can improve the spatial resolution while preserving the spectral information well.