Infrared Image Stripe Non-uniformity Correction Based On Deep Learning

At present,infrared thermal imaging system has been widely used in military and civilian fields,and its core device is infrared focal plane array.Due to the limitations of the manufacturing process and materials,non-uniformity appears in the image,among which the stripe non-uniformity is particularly obvious,which seriously affects the image quality.The traditional method of stripe non-uniformity correction used complex prior knowledge of the image,and the corrected image had artifacts and edge blurring.In this paper,deep learning method is used to correct stripe non-uniformity in real time,and the thermal detail information in the image is kept as much as possible,which is of great significance for improving the accuracy of subsequent processing.This paper will conduct research from the following three aspects:(1)The mechanism of infrared image stripe non-uniformity and its noise model are introduced.Because the parameters of the column-parallel reference pixels,the column-parallel amplifiers and the column-parallel analog-to-digital converters in the readout circuit are different,the stripe non-uniformity is generated under the action of the three,which is appears as striped noise of varying shades;the integrated stripe noise model is used to generate a large number of sub-image with stripe noise and corresponding sharp sub-images,which are used to train deep residual networks.The massive training sub-image ensures the generalization ability of the deep residual network on images with real stripe noise.(2)By analyzing the current deep learning denoising network,an improved stripe non-uniformity correction network based on DnCNN is proposed,and it is trained using supervised learning strategy,the test results show the effectiveness of the improvement.In its structure,the ReLU activation function after the input convolution layer is removed to reduce the loss of simulated stripe information during training;In the middle layer,through downsampling and upsampling operations to increase the receptive field of the network and reduce the amount of calculation,so that the network can make full use of the image context information;A skip connection is introduced between the downsampling and upsampling operations to compensate for the information lost during the image downsampling process,and at the same time fuse the features extracted from different receptive fields of the network;At the output of the network,the convolution layer is added to improve the prediction ability of the residual image;Increasing and decreasing the number of filters in a symmetrical manner enables the network to learn richer residual information.(3)In the loss function,the adversarial loss is introduced,the semi-supervised learning strategy is used to train the improved network,so that the reconstructed image is closer to the sharp image.Optimizing the pixel-level loss function between the reconstructed image output by the network and its corresponding sharp image,such as the mean square error or L1,because it focuses on pixel-level differences,it is easy to make the network smooth the image.In order to better improve the correction effect from the pixel-level loss,a discriminator sub-network is introduced,which together with the improved network forms a generative adversarial neural network.The improved network acts as a generator,and its output reconstructed image better retains thermal details.The method proposed in this paper is tested on images with real stripe noise,and compared with the latest stripe non-uniformity correction method in qualitative,quantitative and running time.The results show that the proposed method has the advantages of real-time correction without artifacts,good detail retention,and no need to adjust specific parameters.