Research On Infrared And Visible Image Fusion Algorithm Based On Deep Learning

Multi-source image fusion is to effectively extract and integrate the complementary information in multi-type image data,eliminate the redundant information,and generate a fusion image with richer information and more accurate description of scene information.Multi-source image fusion technology is helpful to solve the problem of insufficient information of single type image and improve the efficiency of multi-source image data processing.At present,multi-source image fusion technology has been widely used in military,remote sensing,monitoring and medical image fields.Among many kinds of images,infrared and visible images are most widely used in the field of image processing,and they have strong information complementary ability.By fusion of infrared and visible images,the rich detailed texture information and the highlighted target information can be fused to obtain a clear and accurate description of the scene content.The fusion of infrared and visible images is one of the important research directions in the field of multi-source image fusion.In this thesis,infrared and visible image fusion technology is studied.By combining the advantages of deep learning in image processing,this thesis proposes two fusion algorithms for infrared and visible images based on self-supervised learning and convolutional neural network:(1)An image fusion algorithm based on residual network and attention mechanism is proposed in this thesis.In this method,the infrared and visible images are firstly decomposed into the background layer containing low-frequency background information and the detail layer containing high-frequency detail texture information by using guided filtering,and then the feature information is extracted from the background layer and detail layer respectively and fused by using encoder network.Then,the fused image is reconstructed by decoder network.In the decoder network,this thesis improves the residual unit by adding the attention feature fusion module to improve the image fusion effect.In view of the different influences of source images with different information content on the fusion image,this thesis designs an adaptive weight calculation method based on gradient information and uses it in the loss function to adjust the influence of infrared and visible images on the fusion image.(2)An image fusion algorithm based on multi-scale boosted network is proposed in this thesis.On the basis of the above method,this thesis further introduces a multi-scale mechanism.The subsampling operation is adopted in the encoder network to obtain the multi-scale feature information of the source image,and the deconvolution operation is adopted in the decoder network to carry out the up-sampling operation to ensure the size of the output image remains unchanged.In order to reduce the loss of feature information in the sampling process,the multi-scale feature fusion module is used in this thesis to supplement the multi-scale feature information generated in the encoder network to the decoder network.In this thesis,the feature projection module is used to fuse the feature information of different scales before and after sampling,so as to realize the error feedback correction of the feature information and boosted the image restoration and reconstruction effect.Through the above improvements,the method has achieved better image fusion effect.In this thesis,a large number of experiments and comparative analysis of the proposed image fusion algorithm are carried out on the public data set TNO.The experimental results show that the proposed image fusion algorithms perform well,and achieves good fusion performance.