Research On Infrared Image Colorization Based On Generative Adversarial Network

Infrared image colorization is to convert infrared images into colored visible light images,so as to better understand and analyze the information in infrared images.Infrared image colorization is widely used in medical,industrial,and security fields.The infrared image colorization method based on the generative confrontation network is the current mainstream method.According to the difference between supervised and unsupervised technical levels,the infrared image colorization method based on the generative confrontation network can be divided into paired infrared image colorization and unpaired infrared image colorization.Infrared image colorization,the main difference between the two is whether a pixel-aligned paired image dataset is used in training.Existing methods have achieved good colorization effects,but the colorized image still has a gap with the real image in terms of color details and image structure,and there are certain artifacts and distortions.Aiming at these problems existing in the existing methods of paired and unpaired infrared image colorization,this thesis proposes a saliency map-guided Dense Unet generative adversarial network and a Cycle Swin Transformer generative adversarial network.The main work of this thesis is as follows:(1)Aiming at the gap between the color and structure and the real value in the colorization of paired infrared images,this thesis proposes a saliency map-guided Dense Unet generative adversarial network.The existing best method for colorizing paired infrared images.The colorized image still has a gap with the corresponding real value,which is mainly reflected in the image color details and image structure.In terms of image color details,the colorized image is different from the corresponding Some areas of the real image have inconsistent colors.In terms of image structure,the colorized image is different from the texture of the corresponding real image.This thesis proposes a saliency map-guided Dense Unet generative adversarial network to narrow such color and structure differences.The proposed method uses Unet,global feature module,attention prediction module and dense connection module to design the generator of conditional generative adversarial network.Among them,the global feature module enables the model to obtain global semantic features,the attention prediction module guides image colorization by generating saliency maps,and the dense connection module enables the model to obtain deeper features.At the same time,the proposed method also adds frequency domain loss and structural similarity loss to improve the performance of the model.The experiment uses the KAIST dataset to conduct a large number of comparative experiments and ablation experiments.The proposed method is visually superior to the existing methods.When the dataset image size is 256×256,the peak signal-to-noise ratio is increased by 0.6508 d B,and the structural similarity is increased by 0.0218..In addition,the proposed method is also experimented on the official registration subset of the FLIR dataset and visually outperforms existing methods.(2)Aiming at the artifacts and distortion problems in the colorization of unpaired infrared images,this thesis proposes a Cycle Swin Transformer generative adversarial network.The existing best method for colorizing non-paired infrared images The colorized image will produce artifacts and distortions,that is,the artifacts can be seen from the naked eye in some areas of the colorized image,and the image texture in some areas is different from that of Real textures don’t match.This thesis proposes Cycle Swin Transformer Generative Adversarial Networks to reduce artifacts and distortions.The proposed method redesigns the generator of Cycle GAN using Swin Transformer and convolution,and modifies the activation function of the discriminator at the same time.The design of the proposed method combines the advantages of Transformer and convolution to improve the mapping ability from infrared image domain to RGB image domain.In addition,a dataset called NIR2 RGB was collected for the task of colorizing non-paired infrared images.The experiment uses RGB-NIR Scene,MFNet and the newly collected NIR2 RGB data set to conduct a large number of comparative experiments and ablation experiments.The proposed method is visually superior to the existing methods,and the colorized image of the proposed method has the least artifacts and the least distortion.Meanwhile,the proposed method outperforms existing methods on FID and KID scores.