Research On Deep Learning Based Hyperspectral Image Super-resolution Methods

With the combination of the traditional two-dimensional imagery technology and the spectral technology,hyperspectral imagery is designed to obtain the three dimensional hyperspectral images(HSIs).HSIs are of great spectral resolution.For any pixel in the spatial domain,the corresponding material can be deduced according to its continuous and fine spectral curve.Objects' spatial attribute and material attribute can be simultaneously obtained by the HSI.In this way,HSIs have been widely applied in the crop estimation,mineral exploration,military target detection and identification,environmental disaster monitoring and other military and civil research fields.However,limited by the imagery equipment and complex imagery environment,the spatial resolution of HSIs is still relatively low,which is unable to meet the growing needs of people and also makes a limitation on their applications.Deep analysis of the data characteristics of HSIs and enhancing their spatial resolution are important prerequisite to their accurate interpretation and wide application.However,how to super-resolve the HSIs quickly and accurately without changing their spectral characteristics,namely the spectral separability of the original HSI remains unchanged after being super-resolved,so as not to affect the subsequent processing of interpretation and classification of HSIs,deserves close attention.This dissertation is about how to enhance the spatial resolution with the spectral information being preserved.With the deep analysis of the data characteristics of HSIs,we explored three efficient super-resolution methods,and experiments have been conducted on both synthesized and real-scenario HSIs to validate their performances.The main contributions can be summarized as follows:1.The spectral information of an HSI is a continuous spectral vector composed by tens to hundreds narrow bands provided by each pixel.For the vector,what it matters is the magnitude and direction.Therefore,in the super-resolution process of the HSIs,their spectral information can be maintained by keeping the relative relationship between adjacent bands.Convolutional neural network(CNN)learns the non-linear mapping between training data and tags through multiple hidden layers.Therefore,this dissertation presents an HSI super-resolution method based on CNN.The basic idea is to utilize a CNN model to learn the mapping between the low resolution(LR)spectral difference and the corresponding high resolution(HR)spectral difference.This mapping is generalized and applied to the input LR HSI,and to obtain the corresponding HR spectral difference.The HR spectral difference is utilized to guide the super-resolution process of the input LR HSI.Meanwhile,we applied a spatial constraint on the spectral difference reconstructed HR HSI.The LR HSI generated by the reconstructed HR HSI should be spatially close to the input LR HSI.In this way,the entire process comes into a LR HSI back to the LR HSI,which is a closed and loop proces,and the energy is preserved.In this way,the performance of the algorithm is further enhanced.Experimental results have demonstrated that this CNN based HI super-resolution method can efficiently enhance the spatial information of the input HSI with the spectral information preserved.2.From the perspective of the one-dimensional spectral vector,its spectral information can be well maintained by ensuring the relative relationship between the adjacent bands.The starting point of the spectral dimension does not affect the spectral information.However,when comes to the spatial dimension,the selection of the starting point will affect the spatial reconstruction,making the error-prorogation in spatial domain.To solve this problem,this dissertation presents an HSI super-resolution method based on the deep-learned spectral difference and spatial error correction model.The basic idea is to select a key band firstly.When compared with the other bands,the key band is the easiest to be super-resolved,and the super-resolved key band is most close to its reference band,which provides a precise spatial starting point for the other bands.Meanwhile,there may be spatial discrepancy between the deep learned spectral difference and the ground truth spectral difference.Directly utilizing the deep learned spectral difference for super-resolving the rest bands proves to make a spatial error accumulation as the band further from the key band.In order to correct accumulated error,we also propose a spatial error correction model,which multiplies the deep learned spectral difference by a constant,so as to correct spatial error propagation without affecting the spectral information.Experimental results show that the proposed spatial error correction model can achieve good super-resolution performance on both synthetic and real HSIs.3.Limited by the high acquisition cost,it is challenging to obtain a great deal of HSIs,making a limitation of the deep learning based HSI applications.For HSIs,each spatial pixel corresponds to one spectral curve.One HSI usually contains dozens or millions spectral curves.The CNN with spectral curves as training data can avoid the performance limitation caused by the data insufficiency.This dissertation presents an HSI super-resolution method based on one-dimensional spectral mapping convolu-tional neural network(SMCNN)and non-negative matrix factorization(NMF).First,we select the bands in the HSI with a constant interval and super-resolves the selected bands.The remaining bands are interpolated to obtain a initialized and completed HSI.Each pixel in the initialized HSI corresponds to a spectral vector.At the same time,the SMCNN is used to learn the mapping between the initial spectral vector and the ground truth spectral vector.Depending on the SMCNN,the initialized HSI is spectral corrected and an HR HSI is obtained.In addition,to make full use of the information carried by the input HSI,the input LR HSI is fused with the SMCNN obtained HR HSI,which obtains a new HR HSI with more information.Experimental results and data analysis have demonstrated the effectiveness of this method in terms of both spatial information enhancement and spectral information preservation.