Face Recognition Based On Multispectral Image Fusion

Face recognition technology has found wide applications such as human identification, security surveillance, intelligent human machine interface, etc. However, face recognition based only on the visual light spectrum has shown difficulties in performing consistently under uncontrolled operating conditions. Face recognition accuracy degrades quickly when the lighting is dim or not uniform. The use of thermal infrared (IR) images can improve the performance of face recognition under uncontrolled illumination conditions. But eyeglasses may result in loss of useful information around the eyes in thermal face images since glass material blocks a large portion of thermal energy, which are important features for recognition. This paper studies the methodology of fusion visual light and thermal images for improving the performance of face recognition system.In this paper, a new pixel-level visual light and thermal IR image fusion method in the wavelet domain is proposed for enhancing robustness of face recognition, by exploiting the relatively lower sensitivity of visible imagery to occlusions by eyeglasses. The low frequency coefficient visual light and IR images are combined with weighted averaging and are sharpened. The high frequency coefficient images are double thresholded and then combined, which not only preserves most significant information of original images but also removes the noise and useless details for face recognition. To evaluate and compare the proposed fusion algorithm, four objective fusion image evaluation indices and face recognition rate are used. The face recognition performance is used as final evaluation index. Experiments results show that the proposed fusion algorithm outperforms other fusion algorithms based on averaging, Laplacian pyramid, and wavelet fusion method using local energy. And Comparison results show that decision fusion-based face recognition techniques outperform individual visual and thermal face recognition.Decision fusion method for face recognition is also investigated in this paper. A prior knowledge about the performance of individual visual light and infrared face recognition is used in the decision fusion process. The probe image is tested with the visual light gallery and the infrared gallery respectively. The decision fusion is to select one of the recognition results. If the probe image is with eyeglasses, the visual light recognition result is always selected. When there are no eyeglasses in the probe image, if the smallest normalized distance between the probe image and the visual light gallery equals to that between the infrared galleries, the infrared recognition result is selected.