Infrared And Visible Image Fusion Based On Generative Adversarial Network

The purpose of image fusion is to combine two or more images to generate an image with complementary information to improve the perception ability and obtain more comprehensive and accurate target information.Infrared and visible image fusion is its important branch,infrared image contrast,not easy to be affected by harsh environment;Visible image texture details are clear,more in line with the human visual system.Therefore,by fusing infrared images with visible images,it is possible to highlight the target information while retaining texture details and obtain images that are more in line with human needs.At present,more and more image fusion algorithms are proposed,but algorithms based on Generative Adversarial Network(GAN)still need to be solved urgently: the GAN-based fusion method cannot focus on the perception of image highlight target and texture information,the use of single discriminator leads to insufficient preservation of infrared image information,and the use of convolutional network ignores the image global relationship.In view of the above problems,this thesis studies the convergence algorithm based on GAN,which is mainly divided into the following three aspects:(1)Aiming at the problem that the GAN-based fusion method cannot focus on perceiving the image highlight target and texture information,an infrared and visible image fusion algorithm based on attention mechanism and GAN is proposed,which is divided into two parts: generator and discriminator,the former makes important image features pay more attention through the encoder-decoder module and channel attention mechanism,thereby improving the feature extraction ability of the algorithm.Generate images and visible images as input to the discriminator to obtain fusion images with richer texture detail.The loss function is divided into adversarial loss function,content loss function and structural similarity loss function,and the parameter setting of structural similarity loss function is explored.The experimental results show that the correlation coefficient(CC)reaches 0.5298,the standard deviation(SD)reaches 9.7173,and the qualitative and quantitative analysis are superior to other algorithms.(2)Aiming at the problem of incomplete display of infrared image information using single discriminator,a fusion algorithm for infrared and visible images based on dual discriminator GAN is studied,and the network is divided into two parts: one generator and two discriminators.Because infrared and visible images contain multi-modal information,the generator encodes them separately using an encoder-decoder network and then decodes the extracted features uniformly.In order to adaptively change the sampling position of the source image,a deformable convolution module is introduced in the encoder part.The experimental results show that the spatial frequency(SF)reaches 0.0616,the entropy(EN)reaches 7.3025,and the qualitative and quantitative analysis are superior to other algorithms.(3)Aiming at the problem of ignoring the global context information of images using convolutional networks,an infrared and visible image fusion algorithm based on CMT and GAN is proposed,which is divided into two parts: one generator and two discriminators.In this method,a new encoder is designed to combine CNN and Transformer for feature extraction,so that the trained model can obtain local information and global information at the same time.Then,two stages of fusion are performed in the image fusion strategy to obtain the image fusion feature map.Furthermore,the decoder is used to convert the fusion feature map into a fusion image.Finally,the fusion image is trained against the source image.The experimental results show that the average gradient(AG)reaches 5.2924,and the gradient based fusion performance(Qabf)reaches 0.5361.The qualitative and quantitative analysis are superior to other algorithms.