Research On Applying Convolutional Neural Network Based Super-resolution In Image Fusion

Image fusion aims to integrate multiple images of the same scene into a single image which contains more information.The new image is higher credibility and more suitable for subsequent image process and analysis than any individual one.With the development of sensors,image fusion has been widely used in military,remote sensing,medical and other fields.Due to the constraints of imaging sensor and signal transmission broadband,the resolution of most source images is limited.It is difficult to obtain satisfactory results only by fusion technology.Therefore,in view of latest developments on convolutional neural network(CNN)based super-resolution,research on how to effectively improve the fused image quality has vital theoretical guiding significance and practical value.The main research contents as follows:1.A remote sensing image fusion algorithm based on CNN is proposed.Applying CNN to extract features of low-resolution multispectral image will make full use of the spatial detail information to enhance its resolution;Then enhanced multispectral image and panchromatic image are fused via Gram-Schmidt transform,which makes fusion result more accurate and reliable.2.The Caffe framework is well built under Windows.A CNN for frequency domain super-resolution is trained here.This model inputs low-resolution high frequency components and outputs high-resolution high frequency components,and can establish the end-to-end mapping between them.3.Based on the trained model,a multi-scale CNN based image fusion algorithm is proposed.Firstly,the source images are decomposed into high and low frequency components by the undecimated wavelet transform;Then,high frequency components are enhanced by trained model to generate the high-resolution high frequency components;Finally,the high-resolution high frequency components and low frequency components are fused with certain rules,and the high-resolution fusion image is generated via corresponding inverse UWT.