Research On Infrared And Visible Image Fusion Based On Semantic Enhancement And Global Consistency

As one of the widely used image preprocessing technologies,infrared and visible image fusion can combine the visible rich details and the infrared stable structure to generate more robust images,it is popular in security monitoring,object detection and other fields.Although the image fusion model based on deep learning can overcome the limitations of traditional fusion algorithms in the feature extraction through powerful feature representation capability,it still has the problems of inadequate feature extraction,unrobust feature fusion and unable to achieve real-time performance.In our work,two image fusion models based on generative adversarial network,GC-GAN and AT＿GAN,are designed to solve these problems.Specifically,to improve the feature extraction capability,we designed cross-modal interaction modules in the GC-GAN.In addition,since the global information of the image is conducive to fusing images with good visual quality,we also introduced the graph convolutional network in the feature fusion to capture the global information and improve the fusion effect.In the AT＿GAN fusion model,we use customized feature extraction modules to enhance the features of each mode and design a quality assessment method to guide the network adaptive learning fusion equilibrium point.And,due to the compactness and integrity of the extracted features,it is only necessary to map these features directly to the fused images,which can not only obtain highquality fused images but also achieve real-time fusion.In this paper,the proposed GC-GAN and AT＿GAN are compared with the fusion methods based on deep learning published in recent years,and the great advantages of AT＿GAN in qualitative,quantitative and efficiency analysis as well as the improvement of target detection effect.