Research On Fusion Approaches Of Face Recognition With Infrared And Visible Imagery

Biometric technology provides a highly reliable and robust approach to the personal verification. Among all kinds of biometric technologies, face recognition is a biometric technology possessing great application potential and it is also the one of the most active and challenging tasks for computer vision and pattern recognition. It has a great amount of potential applications in public security, law enforcement, information security, and financial security. Multimodal image fusion for face recognition is the technique that integrates complementary and redundant information of face images provided by multi-sensors to achieve better recognition result. Not only does the technique keep the intrinsic advantages of approaches for face recognition, but fuse useful discriminant information from multi-sensors, which can achieve more accurate and robust recognition performance. However, there are still lots of theoretical and technical problems needed to be solved in this field.This dissertation mainly studies the approaches of multimodal image fusion recognition on feature and matching score level based on the existed theories on infrared and visible image fusion for face recognition. The main work and contributions of the dissertation are as follows:(1)Research on the multimodal face fusion algorithms based Fisher linear discriminant analysis and canonical correlation analysisOn the basis of the ideas of Fisher linear discriminant analysis (FLDA) and canonical correlation analysis (CCA), the paper proposes a fusion method on feature level of pattern classification on multimodal information, called Fisher Linear Discriminant based Canonical Correlation Analysis (FLDA+CCA). The framework of pattern recognition to combine FLDA and CCA is presented. The proposed method extracts feature vectors according to Fisher Evaluation Criterion from two patterns, respectively. Based on the idea of CCA the method establishes the correlation criterion function between the two groups of feature vectors and extracts their canonical correlation features to form effective discriminant vector for recognition. The problem of FLDA projection vectors is avoided when total scatter matrixes are singular, such that it fits for the case of high-dimensional space and small sample size, in this sense, the applicable range of FLDA+CCA is extended. The new method first reduces dimension of the two modals, then builds a correlation between them, which not only eliminates redundant information but constructs the relation among different modals, that effectively employs complementary information to fuse two groups of data for classification. Experiment results demonstrate that the proposed method yields better recognition rate.(2) Propose an algorithm of multimodal face recognition that fusing nonuniform component featuresIn the proposed approach, the original images are firstly divided into modular images,and then pattern features are extracted by linear discriminant method from sub-images, which extends the applicable range of linear discriminant method. Since different facial areas contain variously discriminant information, average facial regions does not effectively reflect the distribution of discriminant information. In order to extract better discriminant local features, the paper employs genetic algorithm to optimize local facial region for feature extraction, then combine holistic and local features for pattern classification. Experiment results of fusion recognition on infrared and visible face images show good recognition performance.(3) Design a two-threshold classifier for fusion of multimodal face informationEnlightened by Dempster-Shaffer evidence theory, the paper presents a two-thresholdclassifier (2TC) to handle multimodal information fusion on matching score level. Based on characteristics of samples in pattern space and Neyman-Pearson criterion, two-threshold classifier, which is derived from the idea of Dempster-Shafer theory, devides pattern space into certain and uncertain region. Fisher linear discriminant criterion is employed to classify samples in the uncertain area. The proposed approach utilizes different rules for sample classification based on the different locations of samples in the pattern space, which effectively decreases classification error and increases recognition rate. Experimental results on NDHID and Equinox face database show that new method achieves good performance.