Research On Feature Conversion Between Visible Light And Infrared Images

Visible image is suitable for human to observe,and infrared image can represent the information about scene temperature.The imaging spectrum and principle of the above two types are different,but both have important application value.Traditional infrared image simulation methods suffer the problems of low accuracy and complicated modeling process.And the resources of visible images are abundant.Therefore,this dissertation studies a way to directly perform translation from visible images to infrared simulation images,which has the advantages of high efficiency,low cost and high precision.The main work of this dissertation is as follows:(1)A Two-level Light-weight Multi-Scale Information Fusion Generative Adversarial Network(TLMIF-GAN)is proposed.TLMIF-GAN adopts a"from coarse to fine"structure by a cascaded two-level network,which can realize the feature translation between visible and infrared image in the"from coarse to fine"style:the first level mainly focuses on the global structure feature of visible/infrared images by a generation network of the large receptive field;The second level uses a generation network of small receptive field,focusing on processing the local details information of visible/infrared images.(2)A method of combining auxiliary tasks with cascaded adversarial networks is proposed.The contours and semantics of visible and the corresponding infrared images are similar.Therefore,we add the auxiliary task of semantic segmentation to the first-level network to obtain more accurate global structure information.Based on the coarse results which come from the first level,an auxiliary task is added to the second-level network,which converts the coarse results of the second-level to the grayscale-inverted visible images to supply local information.(3)The optimization methods for accuracy and efficiency are applied in TMIF-GAN.For better performance of the translation,a Multi-scale Fusion Module(MFM)is proposed,which can improve the overall network accuracy by fusing multi-scale feature information under different receptive fields;secondly,to reduce the parameters of the network,a variety of lightweight convolutions are compared and analyzed.And group convolution and Ghost Module is adopted in the final network;finally,a displacement fields network is designed to further optimize the results,and the optimized results are slightly improved in objective evaluation metrics.This dissertation performed experiments on the public dataset of Multispectral Pedestrian Dataset(MPD)and the indoor near-infrared dataset.Our method is superior to other advanced algorithms in a variety of objective metrics and computational efficiency.Among them,our method is about 8.41%?28.96%?11.73%?9.75%?1.14%higher than Pix2pix,X-Fork,Select GAN,SEAN,TMIF-GAN respectively in the accuracy metric P_?<1.25.Moreover,the structural and texture features of the translation results of our method are relatively correct,which means that the translation results are better than other advanced algorithms in subjective visualization.