Infrared And Visible Image Fusion Based On Deep Learning

Image fusion is a kind of image enhancement technology.The infrared and visible image fusion investigated in this paper has always been a hot topic in the field of image fusion.The infrared image shows the thermal radiation information of the target object,which has a strong indication effect on the blocked object,but it is fuzzy for most of the details.Visible light image has high resolution and clear scene texture,but it is not very indicative of the target object due to its rich details.The two kinds of image fusion make use of the correlation of infrared and visible images in time and space and the complementarity of information,to highlight the target object while retaining the details of the background,which helps the system to work at any time without the influence of time and weather,so that the fused image is more conducive to the recognition of human eyes and the machine to achieve target detection,tracking,monitoring and other tasks.The traditional fusion method of infrared and visible image is developed early and widely used.However,in image fusion,the results often have the shortcomings of low contrast and spectral distortion,or the problems of large computational intensity and redundant information generating block artifacts.With the continuous development of deep learning,many researchers have applied some deep learning methods to infrared and visible image fusion.In the process of deep learning,the multi-layer neural network is trained to learn the complex relationship between data under the constraint of loss function.At the same time,the attention mechanism is added,so that the fusion results can effectively extract features without producing artifacts,and the authenticity and details of the image can be retained to a large extent,so as to achieve better results.This paper proposes two different methods for image fusion of infrared and visible light,aiming at three aspects: simplicity of computation,authenticity of fusion and excellence of results,combining skip connection,residual block,attention mechanism and smoothed dilated resblock(SDR).Its main work includes:1.An infrared and visible image fusion method based on residual network and attention mechanism is proposed.In order to better fuse the information contained in the source image,the feature extraction phase includes three layers of normal convolution and three residual modules,which can better extract shallow features and deep features;in the feature fusion stage,an attention mechanism is added to obtain an attention map to fuse the deep features;the feature map obtained from the first two layers,i.e.the shallow features,is passed to the corresponding deconvolution layer for processing through a skip connection;finally The fused image is obtained by constraining the network with pixel loss and SSIM loss function.2.A fusion method of infrared and visible images based on SDR and Two-channel attention mechanism is proposed.SDR module is an improved residual module,which can make the network ignore additional parameters in the training process,expand the sensitivity field of the convolutional layer without generating grid artifacts,effectively extract depth features of different scales,and reuse shallow information through skip connections to retain more meaningful information for the fusion task.In the feature fusion stage,channel and spatial attention mechanism is added to fuse the obtained features.Finally,the SSIM and MSE loss functions were used to constrain the mean square error and structural similarity loss of the network,and the fusion image was obtained.