Research On Key Technologies Of Face Image Enhancement And Recognition Based On Shortwave-infrared Imaging System

Due to the unique spectral characteristics of short-wave infrared radiation(SWIR),short-wave infrared imaging systems have the advantages of capturing image information at night and less disturbed by light changes.In recent years,in order to improve the quality of images acquired in low-light environments such as nighttime and ensure that security and monitoring equipment can truly work around the clock,short-wave imaging systems have become a new promising direction of development.But also because of its special spectral characteristics,there is a large modal difference between the image acquired by SWIR imaging system and the common visible infrared image,which affects the usage of SWIR images.In monitoring and security applications,the identification of people in images is a basic requirement.The enrolled facial images stored in most systems are all visible light images.The modality gap between short-wave infrared images and visible light images makes it difficult to recognize human faces according to short-wave infrared human facial images.With the development of deep learning technology,deep neural networks have made breakthroughs in various fields of image processing,but the research on SWIR facial images is limited by the amount of data and the difficulty of acquisition.In this paper,a short-wave infrared-visible light facial image dataset is established.Using deep learning schemes,the key technologies of short-wave infrared facial image enhancement and recognition are studied,covering the translation between short-wave infrared facial images and visible light facial images,short-wave infrared facial image inpainting and short-wave infrared facial image recognition according to enrolled visible light facial images.The main work and innovations are as follows:(1)We establish the Shortwave-Visible light Face Dataset(SVFD).We analyze the problem of enhancement and recognition of shortwave infrared-visible face images and summarize the requirements to be met by the research data.Then,we acquire and process the images to establish a shortwave infrared-visible face image dataset SVFD.Also,we test some common face recognition algorithms on SVFD and illustrate the usage and the characteristics of SVFD.(2)We study the translation between short-wave infrared facial images and visible light facial images.New loss function calculation paths are added and the relation between SWIR and visible light facial images are taken into concern.The structural information of images is used and the two training paths of Cycle GAN are connected by new loss function calculation path.The proposed framework solves the problem that the network may learn the wrong mapping during training process,and can translate the short-wave infrared facial image into a visible light facial image closer to what the human eye sees daily.(3)We study the SWIR facial image inpainting and improve the Deep Fill framework to solve the problem of poor repair results and blurry areas in filled images.The dilated convolution module in Deep Fill is modified to expand its effective receptive field.A new loss function according to the similarity of images is applied and the framework this paper propose can output images with more realistic filled area.(4)Due to the modality gap between short-wave infrared facial images and visible light images,it is difficult for general(visible light)face recognition algorithms to accurately recognize SWIR facial images according to visible light facial images.However,most of the enrolled images widely used now are visible light facial images.A cross-modality face recognition algorithm based on content feature extraction is proposed.The “content extractor” from image translation framework is introduced to overcome the modality gap in recognition.DRIT—an image translation framework based on disentangled representation—is improved to get better content extractors.The content extractors from the improved DRIT framework can get more accurate content features from input images.A network used to process and identify the face according to the content features is designed and trained.The accuracy of SWIR-VIS face recognition is improved.