Research On Infrared Image Super-resolution Technology Based On Convolutional Neural Networ

The breakthrough of sensor resolution has always been focused on the computer vision area.With the increasing demand for high-quality infrared images,it is urgent to break through the resolution limit in infrared imaging field where the progress of sensor technology is slow.At present,most of the super-resolution algorithms based on convolutional neural network use paired datasets,resulting in the poor visual perception of super-resolution images.To solve this problem,this paper proposes an unsupervised infrared image super-resolution algorithm that does not depend on paired datasets to improve the visual performance of infrared image superresolution.Aiming at the problem that the super-resolution network is difficult to overcome the resource constraints and realize the platform deployment,this paper compresses and accelerates the super-resolution network,and designs and constructs a system that can realize the real-time super-resolution of infrared images based on the embedded platform.The main research contents of this paper are as follows:(1)An unsupervised super-resolution algorithm for infrared images based on generative adversarial network is proposed.A double discriminator structure is designed to cooperatively constrain the authenticity of the generated image and the performance of texture detail generation.Unlike supervised super-resolution algorithms,which relie on paired datasets to learn the mapping from degraded images to original images,this algorithm does not rely on paired datasets,but directly maps the original images to higher resolution dimensions by learning their own characteristics.This algorithm can break through the sensor resolution limit,overcome the problems of image over smoothing and network over fitting caused by relying on paired datasets,and improve the visual perception of human eyes.At the same time,this algorithm has strong applicability and transferability.The supervised super-resolution network,which originally needed to use paired datasets in the training process,can be substituted into the generator subnetwork to achieve unsupervised learning and improve the visual performance of super-resolution images without relying on paired datasets.In this paper,a simulated dataset is created and the rationality of the algorithm is verified by ablation experiments.An infrared image dataset is created,and compared with other algorithms.It is verified that the proposed algorithm can generalize better when the sensor resolution limit is exceeded and present more realistic infrared super-resolution images.Compared with the typical super-resolution algorithm to improve the visual perception of human eyes,the authenticity difference and detail difference of the image are optimized by 27.05% and 15.87% respectively;Compared with other new super-resolution algorithms that do not rely on paired datasets,the variance,edge strength and information entropy are increased by 4.02%,17.64% and 0.25% respectively.(2)Based on the embedded platform,a real-time super-resolution system of infrared image is built.Aiming at the problem that it is difficult to implement algorithm deployment on the platform with limited resources,ESPCN＿WS is created based on the lightweight superresolution network ESPCN,and structured pruning and perceptual quantization training is used to realize network compression and acceleration.Moreover,the inference acceleration engine Tensor RT is loaded on the NVIDIA-Jetson-AGX-Xavier core board to further improve the network inference speed.After verification,the system generates the infrared image with the resolution of 640*512 into the super-resolution image with the resolution of 1280*1024,which can achieve the processing speed of 30 frames per second,ensure the real-time performance and have a good super-resolution performance.