Research On Infrared And Visible Light Image Fusion Method Based On Multi-scale Statistical Modeling And Deep Learning

Image fusion refers to the fusion of two or more source images into a target image through a certain algorithm.The main purpose of obtaining the fused image is to retain the useful information of the source image and improve the visibility of the image.Because the fused image has high quality and higher observation clarity,it is more convenient for people to study the image information.Visible and infrared image fusion is an important part of image fusion related research,which meets many military and civil needs to a great extent.In addition,infrared and visible image fusion technology has portability and can be extended to other types of image fusion fields.It is helpful to further improve the theoretical system of image fusion.The research methods of infrared and visible image fusion usually include methods based on spatial domain,transform domain and depth learning.Among them,the fusion method based on multi-scale transform is widely used in the field of infrared and visible image fusion,including wavelet transform,contourlet transform and non down sampled contour wave transform(NSCT).In recent years,the research of infrared and visible image fusion based on deep learning has developed very rapidly.Common deep learning models include generative countermeasure network(GAN),convolutional neural network(CNN)and pulse coupled neural network(PCNN).At present,the research in this field has become the focus of attention at home and abroad,and its research results can promote the wide application and substantial development of this field in real life.Firstly,this thesis introduces the research status and development trend of image fusion at home and abroad in detail,and summarizes some classical image fusion methods.Then,the multi-scale transformation theory and deep learning technology are introduced.Finally,the quality evaluation criteria of image fusion are summarized.Then,the new fusion method of statistical modeling of visible and infrared images in NSCT domain is introduced.Firstly,NSCT transform is used to decompose the source image,and then PCA method and GGD modeling method are used to fuse the low-frequency subband and high-frequency subband respectively.Finally,the fusion subbands are transformed by using the inverse NSCT transformation method,and the fused image is obtained.The fusion results are evaluated by the combination of subjective and objective evaluation mechanism,and compared with the three fusion methods in NSCT domain.Experimental results show that this method has better comprehensive performance.Secondly,an infrared and visible image fusion method combining convolution sparse representation(CSR)and NSCT is introduced.Firstly,NSCT transform is used to decompose the input source image to obtain high-frequency and low-frequency subbands.Then,the high-frequency subband is enhanced by guided filtering,and the enhanced high-frequency subband is fused by the selected maximum strategy.The low-frequency subband is fused by CSR and the selected maximum strategy to obtain the fused sparse coefficient map,and the low-frequency subband coefficients are reconstructed by convolution combined with the learning dictionary.Finally,the fusion subbands are transformed by inverse NSCT transform,and the fused image is obtained.Compared with other latest methods,the fused image obtained by this method has certain advantages both subjectively and objectively.Finally,a new infrared and visible image fusion method based on generated countermeasure network is introduced.Firstly,an Antagonistic Game between a generator and two discriminators is established.The purpose of the generator is to generate a real fusion image and deceive the discriminator with a specially designed generator that reflects the loss of content.The purpose of the two discriminators is to distinguish the structural differences between the fused image and the two source images.During training,MS?SSIM is used as a content loss function.MS?SSIM can maintain stable performance for image fusion with different resolutions,and MS?SSIM can better retain the detail information and contour information of the image.Through antagonistic learning,the fused image can obtain more detailed information.In this thesis,TNO data set is used for training.The experimental results show that compared with the existing typical methods,the subjective fusion image obtained by this method retains more detail information of the source image,has less artifacts,and also has better fusion performance in the objective evaluation index.