Research On Image Fusion Method Based On Separated Representation Learning

Image fusion is the integration of images containing different information captured by multiple sensors in the same scene into a new high-quality fusion image.Compared with the original image,the fusion image has richer information and can play a greater role in the fields of military,environmental monitoring,digital photography and medical diagnosis.According to different fusion strategies,image fusion algorithms can be divided into spatial domain-based image fusion algorithms,transform domain-based image fusion algorithms,and deep learning-based image fusion algorithms.The image fusion algorithm based on the spatial domain first decomposes the input image into small blocks or regions divided according to a certain standard,and then calculates the saliency of the corresponding region,and finally merges the regions with the greatest degree of matching to form a fused image.The image fusion algorithm based on the transform domain transforms the source image into some feature domains through multi-scale geometric decomposition,and then merges the features of multiple input images and inversely transforms them to generate a fused image.Compared with the traditional method,the convolutional neural network has better feature extraction ability and better robustness.Therefore,based on the cognition of shared public features and specific private features in image pairs,studying the separation of image features in the network and designing corresponding fusion rules can improve the fusion effect and obtain fused images with rich information and clear details.The main research contents of this paper are as follows:(1)As an unsupervised learning model,the auto-encoder network reduces the trouble of manual production and annotation of data sets.At the same time,the ability of the autoencoder network to encode and decode information features makes in the field of image fusion,more and more people use the framework to extract features from images,manually design corresponding fusion rules and reconstruct these features to get the final fusion image.However,these algorithms often only extract global features on a pair of source images,and then design complex fusion rules to fuse these features.This easily leads to the loss of information,which in turn affects the final fusion result.Although a pair of infrared and visible images captured in the same scene have different modalities,they also have shared public information and complementary private information.Inspired by residual network,in the training stage,each branch is forced to map a label with global features through the interchange and addition of feature-levels among network branches.What's more,each branch is encouraged to learn the private features of the corresponding images.In the fusion stage,the maximum fusion strategy is adopted to fuse the private features,add them to the learned public features at the decoding layer and finally decode the fused image.The model is trained over a multi-focused data set that synthesized from the NYU-D2 and tested over the real-world TNO data set,experimental results show that compared with the current mainstream infrared and visible fusion algorithms,the proposed algorithm has achieved better results in subjective effects and objective evaluation indicators.(2)On the basis of multi-modal images,the deep auto-encoder network is used to conduct further research on multi-modal images.Based on the cognition of private information and public information,a joint auto-encoder network with parallel structure is proposed to better extract the above-mentioned image information.In the network structure,the encoder is divided into two private branches and two public branches.The private branch is used for learning complementary features,and the public branch is used for learning redundant features.In the feature fusion stage,corresponding fusion strategies are designed to integrate redundant and complementary features into the fusion features.Through the visualization of features,it can be seen that the designed fusion strategies have better integrated image information and decoded good ones.By comparing with mainstream algorithms as GFF,LPSR,GTF,CSR,MFCNN,Deepfuse,GAN,Densefuse and IFCNN methods,the proposed method has rich subjective details and good image structure,which is more suitable for human visual perception and get excellent performance on multiple objective indicators.(3)Further research on joint auto-encoder network,combined with contrastive learning,using cosine function as a constraint in the feature subspace of the image to maximize the distinction between private features and public features.At the same time,in order to make the public features unique,the public branches are combined into one,and a public image containing the source image pair information is used as input,then a pair of source images are input into two private branches respectively.The output of the public branch and the private branch are combined and mapped to two source images to learn the features of the image.Extension of image separation representation learning to multi-focus images,infrared and visible images and medical images.For the extracted features,new fusion rules are designed to adapt to multi-domain image fusion.The proposed algorithm is compared with the current cutting-edge algorithms on the public data sets in the above fields.The experimental results show that the proposed algorithm has achieved good results in subjective effects and objective evaluation indicators.(4)In order to solve the problem that the auto-encoder network cannot generate fusion images end-to-end and reduce the design of corresponding fusion rules,an end-to-end training unsupervised model based on adaptive feature fusion is proposed.The network uses the autoencoder block to extract the features of the image,and directly decodes the image after cascading the features.Because the generated image is constrained by the joint loss function,the details of the source image can be preserved and the structure information and gradient information of the source image can be preserved.The structure of fusion image is clear.Experimental results prove that the model is not inferior to mainstream algorithms and proposed algorithms in subjective effects and objective indicators.