A Unified Image Fusion Network With Multiple Encoders

Image fusion is an important topic in the field of information processing.It aims to extract complete visual information from two or more images with complementary information,and generate an image with rich information and clear edges.The fusion image is more in line with the perception habits of the human visual system,and can provide discriminative features for subsequent computer vision perception.The challenge of the image fusion task is how to retain significant and complete source image information in the fusion image.In recent years,with the rapid development of convolutional neural network technology,image fusion technology has made remarkable progress,but most of the methods are only designed to process a single image fusion task,and do not consider and combine the inherent connections in multiple fusion tasks,which limits the effect of the fusion model in processing the corresponding task and the generalization performance under multiple tasks.This article discusses and verifies the shortcomings of the model learned under a single task data in dealing with a single task and the limitations while facing multiple tasks.The motivation drives the research to propose a method which learns multi-task cross-domain knowledge to improve the robustness and universality of the fusion model.In order to learn and apply multi-task cross-domain knowledge,this paper designs a unified image fusion framework.First,this paper designs a multi-encoder image fusion model based on convolutional neural networks,which consists of the following three parts: a multiencoder network containing multiple task-oriented encoders,a feature fusion module and a generic decoder network.Each task-oriented encoder network learns specific fusion task domain knowledge,which is used to extract specific features in each task data.The fusion layer module fuses the features output by multiple feature extractors to obtain more informative features.Synthesize cross-domain knowledge through a generic decoder and output a fused image.In addition,this paper proposes a model alternate optimization strategy and a selfadaptive loss function to use multi-task cross-domain knowledge to train the model flexibly.Finally,in order to improve the saliency of the features in the spatial domain and the channel domain,this paper combines the non-local network to design the non-local channel densely connected network and the regional spatial attention network to explore the potential correlation between channels and the long-term dependence between features.This article has conducted a rich and comprehensive experiment,comparing multiple existing image fusion technologies on multiple data sets.The subjective evaluation and objective evaluation results verify the effectiveness and generalization of the method in this paper when learning and processing multiple tasks.Sufficient ablation experiments and analysis verify the role of each method in the unified image fusion framework.