Research On Infrared And Visible Image Fusion Algorithm Based On Deep Learning

Image fusion provides an effective method to enhance and combine pixel-level data.Compared with single input source data,this method provides high-information data for human perception.Infrared and visible light image fusion technology makes full use of the different image characteristics obtained by different sensors,retains the complementary information of the source image during the fusion process,and improves the credibility of the fused image.The infrared image and the visible light image with high resolution and clear scene texture are fused to generate a fused image with obvious target features and clear texture,which has a wide range of applications in production and life.Based on the existing infrared and visible image fusion methods,this paper introduces deep learning into the field of image fusion,and conducts exploration and research in the following two aspects:(1)Aiming at the defect that image features cannot be extracted in depth based on non-deep learning image fusion methods,a multi-scale rolling guided filter and fixed depth pre-trained residual network infrared and visible image fusion method is proposed;the research uses rolling guided filtering and Gaussian filtering is combined to perform multi-scale decomposition of the image,and the source image is divided into a base layer and a detail layer.Compared with traditional multi-scale decomposition,this method has the unique characteristics of retaining specific scale information and reducing edge halo.The Residual Network is used to obtain the depth features from the source image,the initial weight map is obtained through the l1 norm and smoothing operation,and the final weight map is obtained using the softmax algorithm.The weighted fusion of the weight map and the base layer is used to obtain the base layer fusion image.The detail layer uses the combination of taking the largest absolute value of the coefficient and Gaussian filtering for weighted fusion.Finally,the fusion image is obtained through the inverse transformation,and the experiment proves that the algorithm achieves a good fusion effect.(2)Although the pre-trained Residual Network can extract image feature information in depth,it lacks specificity for infrared images.Therefore,a fusion method of infrared and visible light images based on similarity learning in the Siamese Network in the multi-scale directional local extremum decomposition domain is proposed.The research uses the combination of local extremum filtering and Nonsubsampled Directional Filter Banks to decompose the source image in multi-scale direction to obtain the base layer and the detail layer.Since the fixed network training uses natural image sets,it is not necessarily optimal when extracting infrared and visible image features.We initialize the Siamese full Convolutional Neural Network with the pre-trained structure learned from natural data,and use transfer learning.Method to learn the similarity measure based on cross-correlation.The training data set is a pair of positive and negative detail coefficients decomposed by multiple scales and multiple directions.The base layer uses the visual saliency map of the source image as a weight map for weighted fusion.The detail layer is sent to CNN to extract the feature map,and then the weighted normalized cross-correlation matrix of the detail layer and the feature map is calculated,and it is used as the weight map to fuse the detail layer image.Finally,the fusion image is obtained through inverse transformation.The experimental results show that the algorithm proposed in this paper has a better performance,and the subjective and objective evaluation results are the best compared with the classical methods.