Research On Cross-modal High-resolution Image Reconstruction Technology Based On Deep Learnin

From the "debut" of the first film camera to the "ubiquity" of smartphones,the image sensing technology with the optical signal as the carrier integrates optoelectronics,information optics,and machine vision.The visible imaging detector has been widely used in various photoelectric imaging systems with its advantages of seeing and human-like imaging.On the contrary,the infrared imaging detector represented by thermal radiation is used for penetrating smoke imaging,and the spectral characteristics of sensing temperature give it many unique advantages that are difficult to obtain or even impossible to obtain in the visible spectrum.Combined with the new theories of information optics and machine vision,this dissertation aims to break through the bottlenecks such as insufficient spatial sampling of the infrared detector and limited information obtained by a single sensor,improve the resolution of infrared image and improve the quality of infrared visible light fusion image through convolution neural network,and try to provide a new idea for the realization of the cross-modal high-resolution photoelectric imaging system.The main contents and innovations of this paper are as follows:(1)Aiming at the problem of insufficient spatial sampling and low image signal-to-noise ratio of infrared imaging detector,a convolution neural network image super-resolution reconstruction method based on symmetrical skip connection is proposed.The network structure is mainly composed of convolution layers and deconvolution layers.The features are extracted through the convolution layer,and the super-resolution reconstruction is carried out through the deconvolution layer.In the network structure,the convolution layer with strides of2 is used to reduce the dimension of the feature maps,which is conducive to the network to extract the feature information at different scales.In addition,the skip connection and residual blocks are introduced into the network structure to solve the gradient disappearance in the traditional network.The number of feature maps is increased by adding the number of channels,so the super-resolution reconstruction ability of the deconvolution layer is improved.The network structure combines global residual learning with local residual learning to improve the overall performance of the network.The experimental results show that subjectively,the improved infrared super-resolution reconstruction algorithm effectively enhances the highfrequency details of the infrared image,and the edge details of the reconstructed infrared image are clearer.From the objective data,when the upper sampling scale is ×4,the average peak signal-to-noise ratio of the image is 1.31 d B higher than VDSR.(2)Aiming at the problem of low fusion image quality caused by different imaging mechanisms and spatial resolutions of different modes,an infrared and visible image fusion network structure based on deep learning is proposed,which adopts a fusion encoder-decoder network structure for end-to-end learning.The multi-scale feature extraction block and the channel attention block perform deep feature extraction for the input infrared image and visible image respectively,and the fused image of both is obtained by the elemental-max fusion method.The encoder-decoder structure performs super-resolution reconstruction of the fused image,down-sampling the feature map using the convolutional layer with strides of 2,up-sampling the feature map using the deconvolutional layer,and recovering the detailed information of the image features.The skip connection in the structure transfers the image feature information from the encoder part of the network to the decoder part and also solves the gradient disappearance.In addition,a hybrid loss function is designed to transform the image fusion problem into a structure and intensity ratio maintenance problem for infrared visible images,and to expand the weight difference between the thermal target and the background,so that the fused image can bring out the thermal target better.From the subjective point of view,the fused image is clearer in terms of visual quality and more pronounced in terms of contrast.From the objective data,our method improves 3.6413,1.2493,and 0.2289 in spatial frequency,edge intensity,and average gradient,respectively,compared with the MGFF method.