Research On Key Technologies Of Infrared Image Processing Based On Convolutional Neural Networks

Infrared imaging is a technique that captures infrared radiation from an object and converts it into an understandable visible image for people.In recent years,infrared imaging has been rapidly developed in the fields of security monitoring,medical imaging,forest fire prevention,power failure detection and military reconnaissance,and has received extensive attention.However,limited by the existing manufacturing process,the photoelectric conversion efficiency of each detecting unit in the infrared focal plane array is inconsistent,resulting in serious non-uniformity in the infrared image outputted by the detector.In addition,because the temperature difference between the target and the background is small,and the infrared imaging technology uses the temperature difference between the target and the background to image,and at the same time belongs to grayscale imaging,the infrared image itself has low contrast and single color,which reduces the ability of the human eye to recognize and detect the target.In recent years,deep learning has achieved better results than the traditional method in areas such as face recognition and target detection.So it has a broad prospect for application in infrared image processing.In view of this,our paper focuses on the research of key technologies of infrared image processing based on convolutional neural networks,including stripe noise removal,optical noise removal,contrast enhancement and colorization of single infrared image.The main contents of this paper include:1)We present a convolutional neural network-based method for single infrared image stripe noise removal to solve the problem that the traditional stripe noise removal algorithm cannot preserve the image details.Our method is denoted as a deep convolutional neural network that takes the noisy image as the input and outputs the clean image.It directly learns the end-to-end mapping between images with/without stripe noise.The deep convolutional neural network utilizes convolution kernels of different sizes to combine stripe noise removal and super-resolution operations to achieve the effect of preserving image detail while removing stripe noise.The experimental results show that this algorithm has a simple structure and exhibits excellent image restoration quality and processing speed.2)We propose a robust destriping method based on data-driven learning to address the problem of strong stripe noise.A UNet-like network is used to learn the regularity of complex stripe noise characteristics on a large set of training data,achieving highly nonlinear complex mapping between images with/without stripe noise.To preserve the details of the image in the high-level feature space,a perceptual loss is introduced in the objective function.Experiments show that our network significantly outperforms state-of-the-art destriping approaches.3)We propose a conditional generative adversarial network-based method for single infrared image optical noise removal.With a fully convolutional neural network,it is able to eliminate the optical noise in single infrared image.Our architecture consists of two networks: a denoising network and a conditional discriminator.The denoising network takes a noise image as input and outputs a denoising result,while the discriminator tries to make the output look more like the target.We add different intensity optical noise on the visible image to generate a large number of training samples to enhance the generalization ability of the generator.Significant image quality in experiments is achieved compared with the existing method.4)We propose a conditional generative adversarial network-based method for single infrared image enhancement.Visible images are used for training since there are fewer infrared images.Proper training samples are generated to ensure that the network trained on visible images can be well applied to infrared images.The existing convolutional neural network architectures,such as residual architectures and encoder-decoder architectures,fail to achieve the best results both in terms of network performance and application scope for infrared image enhancement task.To address this problem,we specifically design a new refined generative architecture with skip connections that produces visually very appealing results with higher contrast and sharper details compared to other network architectures.Experiments demonstrate that our approach outperforms existing image enhancement algorithms in terms of contrast and detail enhancement.5)We propose a single thermal infrared colorization method based on conditional generative adversarial networks to transform a thermal infrared image into a realistic RGB image.We propose learning the transformation mapping using a coarse-to-fine generator that preserves the details.Since the standard mean squared loss cannot penalize the distance between colorized and ground truth images well,we propose a composite loss function that combines content,adversarial,perceptual and total variation losses.The content loss is used to recover global image information while the latter three losses are used to synthesize local realistic textures.Quantitative and qualitative experiments demonstrate that our approach significantly outperforms existing approaches.In summary,the five deep learning-based algorithms proposed in this paper effectively solve the non-uniformity,low contrast and single color problems in infrared imaging.