Research On Infrared And Visible Image Fusion Based On Deep Learning

Infrared image mainly contains thermal information of the scene,which is not affected by external lighting conditions and can work all day long.But infrared image has obvious shortcomings such as poor texture and no color information.Visible image contains rich texture information,but which is very sensitive to environmental changes,such as low illumination,fog and other conditions,image quality significantly decreased.The fusion of infrared and visible image can obtain rich detailed information from visible image,obtain significant target information from infrared image and generate high-quality image.However,in the existing fusion methods,the fused image lacks some significant features and the background details are poorly preserved,which is not conducive to human visual perception.Therefore,in order to further improve the fusion effect,the following fusion methods are proposed in this thesis:(1)Aiming at the problems of inadequate extraction of effective information and unclear texture of fused image details,based on self-coding network image fusion methods,this thesis proposes an image fusion method based on mixed attention residual.In order to extract the depth features,this thesis proposes a mixed attention residual module.This module firstly extracts the feature information of multi-channel images.In order to consider the interdependence between the feature map channels,an improved attention mechanism is added.The attention mechanism can capture one-dimensional and two-dimensional features and assign different weights to each channel.The combination of pixel loss,structural similarity loss and gradient loss is used to form a strong constraint of training,which can retain more prominent information and targets.(2)Aiming at the problems that single discriminator can not save enough infrared target and detail information,and the texture information of source image is less,based on generative adversarial network image fusion methods,this thesis proposes an image fusion method based on dual discriminators generative adversarial network.The fusion task of infrared and visible image is completed in the generator.In order to extract rich features,a multi-scale residual module is proposed.By connecting multiple multi-scale residual modules with a bottleneck block,more global features can be extracted.In order to ensure that the fusion image can retain the significant area of the source image,a fusion strategy based on self-attention mechanism is proposed.The image after fusion saves the intensity and detail information of the infrared image through the infrared discriminator,and captures the rich texture information of the visible image through the visible discriminator.The experimental results show that the fusion algorithm proposed in this thesis is superior to the current mainstream infrared and visible image fusion algorithms in subjective and objective evaluation,and retains more significant target information of infrared image and detailed texture information of visible image,and has good fusion performance.