Research On Infrared And Visible Image Fusion Method Based On Weight Map Supervision

Due to the different imaging mechanisms of infrared and visible image sensors,infrared images have prominent targets with high brightness and contrast,but lack detailed information such as texture and edges.In contrast,visible images are rich in texture detail information but lack salient target information.Infrared and visible image fusion(IVIF)technology aims to fuse the features of both images to obtain a single image with high brightness,high contrast and rich texture details at the same time.This technology has a wide range of applications in the military,video surveillance and other fields.Currently,infrared and visible image fusion methods include traditional methods and deep learning-based methods.Traditional methods can accurately fuse the texture details and other major parts of the source image,but the process relies on human-specified fusion rules and has a large computational cost.In contrast,deep learning-based methods can adaptively implement image fusion because it utilizes a large amount of data for training and can automatically extract features from the source image.However,both the existing traditional methods and the deep learning-based methods have some drawbacks.For this reason,two novel infrared and visible image fusion schemes are proposed,aiming to address these drawbacks.The main innovative works include:(1)A dual-supervised weighted graph generation network(DSWGN)-based method is proposed to address the problem of overly complex fusion rules in IR and visible image fusion.The method consists of three parts: a backbone network based on an encoder-decoder structure and two image generation branches.In the backbone network,multiple residual dense involution blocks(RDIBs)are used to extract salient features of infrared images to generate accurate weighted images.Then,the generated weight images are used to define a fusion strategy to obtain fusion results.To solve the problem of not having a reference image for network training,two image generation branches based on Gray Level Co-generation Matrix(GLCM)and Gaussian Blur are designed to generate two types of images focusing on different features of the source image.These two generated images are used to define a joint loss function to supervise the network training.Experimental results show that DSWGN achieves better fusion results in both subjective and objective aspects compared to some state-of-the-art methods.(2)In addition,an image fusion network based on multi-scale a priori information guidance is proposed for the problem of difficulty in weighing the unique features originating from different modal images in infrared and visible image fusion.The network consists of two parts: a backbone network and a content-constrained branch.The backbone network consists of a feature extraction unit and a feature reconstruction unit,in which the feature extraction unit can extract rich multiscale information and generate sufficient a priori information,while the feature reconstruction unit reconstructs the fused image guided by the a priori information.To solve the trade-off problem of multimodal information,a composite loss function constraining image saliency information and detailed texture information is defined to supervise the training of the network.Experimental results show that the proposed network achieves better fusion results in both subjective and objective aspects relative to some state-of-the-art methods.