Research On End-to-end Fusion Of Multi-band Images Via Deep Learning

Simultaneous multi-band detection and imaging of the same scene is one of the main technical features of a new generation of high-precision detection system,which aims to comprehensively utilize the complementary information of different band to obtain a more comprehension and accurate interpretation of the scene.In the existing image fusion research,model-driven fusion method relies heavily on prior knowledge,which results in poor algorithm generalization ability and it cannot be adaptively fused.In contrast,the fusion method based on deep learning has become a research hotspot with its excellent feature extraction ability and high-precision learning capabilities.However,besides the small number of simultaneous fusion,there is also a problem that the final fusion image is not clear enough because of the continuous convolution.To this end,this paper explores a new means to improve fused image clarity combined with attention mechanism and super-resolution on the basis of end-to-end adaptive fusion method.The specific work is as follows:(1)A multi-band images end-to-end adaptive fusion method is proposed.In order to solve the difficult problems in multi-scale geometric tools selection and fusion rules designing in multi-band images fusion,the Generative Adversarial Network(GAN)is constructed.The multi-band source images are synchronously input into the designed residual-based generative network.The network can generate the fused image through adaptive learning.Then,the fused image and the label image are respectively sent to the discriminant.Through the feature representation and classification identification of the discriminator network,gradually optimizes the generator.Via the dynamic balance of the generator and discriminator,the final fusion image is get to achieve end-to-end adaptive.(2)A novel multi-band images adaptive fusion method based on attention mechanism is proposed.Aiming at the low definition and poor details of synchronous multi-band images fusion,the feature enhancement module and feature-level fusion module is designed to improve the quality of fused image details.(3)A multi-contrast super-resolution fusion method for multi-band images is proposed.To solve the problem of unsatisfactory output results caused by the low resolution of the obtained infrared image,a multi-contrast super-resolution subnet is constructed.The low-cost,high-resolution visible image is used to assist in enhancing the infrared image,and the network is combined with attention network to obtain high-resolution fusion image with large amount of information and high definition.