Fundamental Research On Multimodal And Heterogeneous Face Recognition

Automatic face recognition technology has broad applying prospect and promotes the development of human computer interaction and artificial intelligence.With the coming of big data era,there exists both challenge and opportunity in the face recognition field.This thesis focuses on how to embrace the benefit of multi-information to deal with the challenge in the practical face recognition application.The texture feature based on the pixel difference near the landmark is not able to deal with the effect of varying illumination on facial landmark localization task,we need to develop new effective feature to elevate location accuracy;With acquirement of rich multi-modality face images,it is able to utilize the complementary advantages of multi-modality information to improve the performance of face recognition system;classical face recognition algorithm fails to perform a straightforward matching between heterogeneous face image(e.g.photo vs sketch),we need to reduce face variability caused by modality change by heterogeneous face transformation.From the perspective of practical and theoretical,this thesis makes research on three problems: facial landmark localization,multimodal face recognition,heterogeneous face recognition.The main contributions of this thesis are summarized as follows:1)Propose an algorithm to extract facial grayscale geometric feature and fused with texture feature by virtue of cascaded regression model for facial landmark localization task.This thesis presents a grayscale geometric feature based on LTV model to enhance the feature of facial key organs.By virtue of cascaded regression model,it is able to fuse facial geometric information and texture information,generating a model with stronger generalization ability and more robust to illumination change under practical condition.The experiment on two face databases shows that our approach elevates location accuracy and can be applied to real facial landmark localization.2)Propose a feature fusion method of bimodal faces based on Collaborative representationThis thesis proposes the feature representation of bimodal face in the complex domain and fuses the information of vis and nir at the feature level with collaborative representation.Further,sub-block complex Gabor feature combined with CRC(collaborative representation based classification) is utilized to improve the performance of bimodal face recognition system.The geometric error analysis of CRC points out the advantage over SRC(sparse representation based classification).The experiments show the efficiency of our approach.3)Propose a heterogeneous image transformation(HIT)method based on multi-feature fusion via sparse representation at patch level.To address heterogeneous face recognition problem,this thesis proposes a heterogeneous image transformation(HIT)method based on multi-feature fusion via sparse representation at patch level.We first extract local patch structure information of the face image after photometric preprocessing,then gradient information and low-frequency information is utilized to compensate corresponding content of the synthesized image.At last,guided image filtering is used on initial results to enhance fine-scale details with test photo as guidance.The experiments demonstrate that our method is efficient for synthesizing high-definition images robust to expression and pose variations.Moreover,it is able to cope with varying illumination condition.The synthesizing images are competent for later heterogeneous face recognition.All these proposed algorithms have been applied to practical face recognition system.