Research On Infrared And Visible Image Fusion Methods Based On Deep Learning

Infrared and visible imaging technology has been widely applied in many fields such as military,medicine,surveillance,transportation,and power industry.Image fusion aims to combine key complementary information represented by multi-modal images to generate a fusion image whose information is more comprehensive and complete than any individual source image.Therefore,infrared and visible image fusion not only helps to enrich visual perception and facilitate pattern recognition,but also promotes further application and development of infrared and visible imaging technology,which exhibits important theoretical and application value.Aiming at the practical application,this thesis adopts deep learning technology to fully mine the semantic information in multi-modal images and carries on the research of infrared and visible image fusion methods based on deep learning.The main research contents and innovative aspects of this thesis are as follows.Image fusion is an unsupervised learning task without ideal output as a label.Considering the primary difficulty of the constraint designing in deep-learning-based fusion methods,an image fusion method with Generative Adversarial Network and Siamese Network is proposed,which utilizes the global semantic information as a priori knowledge to drive the fusion process.In terms of model design,the structure of fusion network is designed based on residual learning,which improves the non-linear modeling capabilities of the fusion.In terms of model optimization,the structure of discriminative network based on Siamese Network and the discriminant rules based on the completeness of information are designed to achieve high-quality unsupervised learning.In terms of model training,a novel loss function,which is constructed from three aspects: pixel distribution,structural similarity and semantic discrimination,is proposed to effectively constrain the training process.Experimental results demonstrate that the proposed method can generate fusion images with complete and accurate multi-modal information.There is information redundancy in multi-modal images in complex scenes.However,existing fusion methods have limited selectivity and expressiveness for key information,which leads to distortion and inaccuracy in fusion images.To solve this problem,an image fusion method combined with attention mechanism is proposed,which utilizes the key semantic information to improve the fusion performance.On the one hand,to accurately and fully mine key semantic attributes and salient content information of the image,a feature calibration module is built based on the attention mechanism,which assigns channel-wise and spatial attention weights to deep features.On the other hand,to help features achieve propagation and multi-level reuse,an information flow path based on the dense connection is established,which improves the consistency and accuracy of the information in fusion process.Experimental results demonstrate that the proposed method can realize not only the adaptive selection and accurate expression of key multi-modal information,but also a better visual performance in complex scenes.In the end,a combination method with the above advantages is proposed.Experimental results in a substation application demonstrate that visual perception and object detection can be improved by infrared and visible image fusion method,which shows important practical value.