Research On The Infrared Image Enhancement Based On Generative Adversarial Networks

Infrared images often contain a lot of information that is difficult or impossible for human eyes to detect and are widely used in military and surveillance.However,infrared images have limitations like color diversity and resolution,so infrared image enhancement has always been a research hotspot,such as infrared image colorization,infrared and visible image fusion,and infrared image super-resolution reconstruction.The infrared image enhancement methods based on the traditional algorithms often rely on the relationship between pixels,and cannot learn the information and features in the image accurately.Generative adversarial networks(GAN)are optimized by the game between the generator and the discriminator.It can learn the information and features in the images adequately and has great advantages in image generation.Therefore,it is worthwhile to study its application in infrared image enhancement.This paper focuses on two infrared image enhancement problems,namely GAN-based infrared image colorization and GAN-based infrared and visible image fusion.On the study of infrared image colorization,in order to obtain more realistic and detailed results,this paper proposes a DenseUnet GAN structure.An improved generator based on Unet is designed and various loss functions are added to optimize the colorization results.At the same time,the discriminator with deconvolution optimization is designed.The experimental results on the public dataset indicate that the proposed method not only has more realistic effects in visual,but also has higher Structural Similarity(SSIM)and Entropy(EN)than other image coloring methods.The average number of SURF feature points that can be detected from the color images generated by DenseUnet GAN equals to 93.49% of the original visible images,which reflects the good feature reconstruction capability of DenseUnet GAN.In addition,the facial alignment neural network used in preprocessing dataset is accelerated on FPGA.Experiments show that the acceleration method of this paper works 45 times faster than ARM embedded computing on the same board.On the study of infrared and visible image fusion,in order to obtain clear fusion images with more information,this paper proposes an LBP-BE GAN structure.An LBP loss based on the local binary patterns(LBP)algorithm is designed to ensure that the fused images have more edge information.At the same time,a distribution-based discriminator structure is designed to establish the adversarial loss of the generator and the discriminator without ideal fused images as the labels.The proposed method is compared with eight other fusion algorithms on the TNO dataset and INO dataset.The results of LBP-BE GAN not only have clear edges in visual,but also superior to other comparison algorithms in the evaluation of six representative objective indicators.