Research On Infrared And Visible Image Fusion Method Based On Deep Learning

Image fusion technology is one of the important research directions in the field of image processing.In the fields of military reconnaissance,security monitoring,target detection and pedestrian recognition,the fusion of infrared and visible images can improve the observation and detection capabilities of existing technologies,and plays an important role in improving human eye observation and automatic machine recognition.Research on image fusion technology is becoming a hot topic.This thesis studies the problems existing in the current fusion methods based on deep learning.The main contents are as following:This thesis summarizes the current fusion algorithm based on deep learning,analyze its main design ideas and areas that need improvement,so as to determine the main framework of the algorithm.To solve the problem that the fusion images of the current fusion algorithms based on deep learning have poor visual effect and the background details of the source image are dropped in the image,an end-to-end fusion model based on the generative adversarial network framework is proposed.Attention and residual mechanisms are added to the network,and a generator network that can extract features at each level of the image is designed.In order to avoid the loss of shallow features in deep networks,a decoding structure based on u-net connection is used to reuse features.The training method was improved to use pre-fused images as the dataset for training.In addition,the loss function is improved to increase the amount of information retained in the fused image.Experiments on the TNO dataset show that,compared with other fusion algorithms,the algorithm proposed in this thesis achieves certain advantages in both subjective and objective evaluations.A dual-branch network fusion algorithm that preserves image gradient information is proposed.A gradient branch is added to the conventional fusion method,which has only one detail branch.Using gradients as input to convolutions improves the network’s ability to extract features and reconstruct fused images.Then,the proposed network is trained using the MSCOCO dataset.Finally,two common fusion strategies are used to generate fused images,and on this basis,the fusion strategies are analyzed and improved.Experiments show that the proposed algorithm can achieve better objective and subjective evaluations.Improved network structure for unsupervised end-to-end fusion methods.A multi-scale residual convolution module is designed,which uses convolutions of different sizes to expand the receptive field of the network and increase the ability of the network to extract detailed features.In addition,skip connections are added to the network,which enables the network to use features from each level to generate fused images.The structural similarity loss and gradient loss to the source image are added to the loss function,and an ablation experiment is designed to determine the parameters.Finally,experiments on the TNO dataset show that the proposed algorithm can retain more of the source images in the fused images than other contrast algorithms.