Research On The Fusion Algorithm Of Arbitrary Resolution Infrared And Visible Light Images

In the recent years,infrared and visible images fusion is of great significance in various vision-based applications,and it has received more and more attention.However,in most of the existing fusion methods,the input source images and the output fused image are required to have same spatial resolution,which largely hinders the application of these methods in actual scenes.Moreover,when source images have low resolutions,the resolution of the fused image will also be low,that is,the information contained is insufficient.To solve these problems,this paper proposes an infrared and visible image with arbitrary resolutions fusion network based on meta-learning,which can effectively improve and unify the resolutions of infrared and visible images with different resolutions,and fuse them with a novel fusion strategy.The specific method and research result are as follows:1)A meta-learning-based super-resolution and fusion framework for infrared and visible images with different resolutions is proposed.Unlike most existing image fusion methods,the proposed network can receive infrared image and visible image with different resolutions as input,and the model can be used for infrared and visible images with any resolution once the model trained.Thus,the proposed network is of great significance for super-resolution and fusion tasks.2)A novel fusion module based on dual attention mechanism is proposed,which adaptively highlights salient features from the channel and spatial dimension of the feature map,and fuses the feature maps of different source images.This module is designed with an effective fusion strategy to better retain the information of the source image,so that improve the quality of the fusion image.3)A residual compensation module is proposed,which is used to compensate for the lost or distorted information of the input source image feature map during up-sampling,and iteratively cascade could enhance the ability to extract details,thereby enhancing the quality of super-resolution and fusion results.4)By achieving image super-resolution and fusion at the same time,a loss function based on multi-task learning is formulated.This loss function helps the network learn better features,and ultimately improves the quality of the fusion result.In addition,this paper also proposes a novel contrast loss algorithm based on the perceptual color correction theory as a part of the loss function.