Research On Image Fusion Method Based On Generative Adversarial Network

The basic idea of image fusion is to use a specific algorithm to extract useful information of different channels from the images acquired by multiple sensors in the same scene,and finally fuse them into high-quality images to improve the utilization,credibility and accuracy of the images.It is widely used in the fields of monitoring,medical diagnosis and target recognition.In the past few decades,many traditional image fusion methods have been proposed.These image fusion algorithms can be roughly divided into two categories,namely spatial domain algorithms and transform domain algorithms.The spatial domain algorithm first divides the source image into several image blocks according to a certain strategy,and then fuses each pair of image blocks together according to the calculation activity metric.The transform domain image fusion algorithm first decomposes the source image into sub-images with different frequency bands through multi-scale geometric decomposition,then designs the fusion rules according to the characteristics of the decomposed frequency band coefficients,and finally obtains the fused image through the corresponding inverse transformation.In recent years,deep learning technology has been widely used in various computer vision tasks,and has achieved remarkable results in the field of image fusion.Among them,in the field of image generation,the most representative network structure is the generative confrontation network.The adversarial game between the generator and the discriminator makes the quality of the images generated by the generator better and better,and it has been widely used in the field.Therefore,the research on the image fusion method based on the generation confrontation network will improve the fusion effect and get more Accurate and clear fusion images are essential.The main research contents of this paper are as follows:(1)Infrared and visible light image fusion lacks real fusion images.In order to design a generative adversarial network that is more suitable for image fusion,an image fusion method based on weakly-supervised generative adversarial network is proposed.To this end,the saliency map of the source image is obtained through the pre-trained model,and then the saliency map of the source image is fused according to the principle of maximum selection.The fused saliency map serves as the real image of the discriminator,and the saliency map of the fused image The false image that serves as the discriminator,and finally through adversarial learning,can make the fused image contain more salient areas in the source image.In order to improve the fusion quality,a densely connected convolutional neural network is added to the network structure and a perceptual loss is added to the loss function.Experiments on the TNO dataset show that our method better retains the thermal infrared target in the infrared image and the detailed information in the visible light image in terms of objective and subjective evaluation,and is better than the latest methods.(2)The existing infrared and visible light image fusion method based on the generative adversarial network mainly uses the visible light image as the real image of the discriminator,which will cause the fusion image to lose a large amount of infrared target and detailed information in the infrared image,in order to ensure that the source image is at the same time Containing the detailed texture information of the visible light image and the infrared brightness information of the infrared image,an infrared and visible image fusion method based on dual discriminators is proposed.Firstly,the saliency detection is performed on the infrared image,and the salient area in the infrared image is obtained,and then the salient area in the visible light image is obtained by inverting it.Finally,two discriminators are designed to distinguish the salient areas in the source image and the fusion image.Differences,through adversarial learning,can make the fusion image contain more salient areas in the source image.Experiments on TNO data show that the proposed algorithm is superior to the current mainstream infrared and visible image fusion algorithms in objective and subjective evaluation,and it retains the detailed information of the visible image and the infrared target information of the infrared image.(3)We conduct further research on generative adversarial network,combined with the self-attention mechanism,aiming at the problem of ignoring the subjective perception of human eyes during the fusion process of infrared and visible light image fusion methods based on deep learning,and propose a self-attention guide Infrared and visible light image fusion method.In the feature learning layer,the self-attention learning mechanism is introduced to obtain the feature map and self-attention map of the source image.The self-attention map can be used to capture the long-distance dependent characteristics of the image,and the average weighted fusion strategy is designed to fuse the feature map of the source image.Finally,the fused feature map is reconstructed to obtain the fused image.The learning of image feature coding,self-attention learning,fusion rules and fusion feature decoding is realized by generating a confrontation network.Experiments on TNO data show that the learned attention unit reflects the prominent area in the image,which can better guide the generation of fusion rules.The proposed algorithm is better than the current mainstream infrared and visible image fusion algorithm in objective and subjective evaluation.The detailed information of the visible light image and the infrared target information of the infrared image are better preserved.