Research On Low Resolution Face Recognition

Face recognition (FR) has been widely studied for over40years due to its great potential applications. Currently, the advance of technology is able to perform the recognition of complex conditions such as pose, illumination, expression, noise, disguise, aging, race, gender and so on, which is a significant improvement than decades ago. Although the accuracy of face recognition for subjects under controlled conditions can be reached in a satisfactory manner, the performance in real applications such as surveillance remains a challenge. The low-resolution (LR) of the images caused by unperceived surveillance where subjects are far away from the cameras and face regions tend to be small is the major issue, which is classified as low-resolution face recognition (LR FR) problems in the area of study.In this dissertation, LR FR algorithms were systematically investigated for the purpose of solving small size and low quality problems brought up by surveillance at-a-distance. A comprehensive literature review on LR FR algorithms was conducted. Three key issues about LR FR system, namely the limitation of resolution-robust feature representation, the deficiency of unified feature space between LR and HR, and the inconsistency between enhancement and recognition based on super-resolution, were explored in depth. More importantly, a set of new models and algorithms were developed and proposed in the chapters. The major contributions of this dissertation are summarized below:(1) A comprehensive literature review was conducted for LR FR systems, which is the first attempt in this area. First, it gave an overview on LR FR, including concept description, system architecture and algorithm categorization. Second, many representative algorithms were broadly reviewed and discussed. The algorithms were classified into two different categories, super-resolution for LR FR and resolution-robust feature representation for LR FR, which are further classified into four small groups, namely vision-oriented&recognition-oriented super-resolution and feature-based&structure-based resolution-robust feature representation. Their strategies, advantages and disadvantages were discussed in detail. Some relevant issues such as databases and evaluations for LR FR were presented as well. By generalizing their performances and limitations, promising trends and crucial issues for future research were summarized. (2) A new feature-based resolution-robust feature representation algorithm was proposed, namely a graph embedding algorithm (FisherNPE) based on resolution level difference probabilistic similarity measure, which aims to improve the limitation of resolution-robust feature representation. The proposed algorithm includes two modules: feature extraction based on FisherNPE algorithm and feature classification based on resolution level difference probabilistic similarity measure. The feature extraction module buids a new relationship weight matrix by using a weight factor to combine the global and local relationship representation ability of LDA and NPE respectively, which is embeeded into a new graph embedding model FisherNPE, thus realize resolution-robust feature extraction. The feature classification module creatively introduces the concept of resolution level difference combining with the probabilistic similarity measure to build the feature spaces of the differences between HR and LR face images. It would expand the traditional way of "one-to-one" into "many-to-many" in feature classification and greatly incease the amouts of the effective features. Finally, the proposed algorithm was comprehensively tested on three public databases namely ORL, YALE, CMU PIE with the cases of single resolution and multiple resolutions. Experimental results showed that the proposed algorithm had improved recognition performance by10%, especially on the very low resolutions such as7×6.5×5.8×8. in comparison with the traditional algorithms such as LDA, NPE, BayesianFace. And the system maintains a relatively stable performance.(3) A new structure-based resolution-robust feature representation algorithm was proposed, namely a LR and HR feature space match algorithm based on kernel coupled cross-regression (KCCR), which aims to improve the deficiency of the unified feature space. The idea of kernel coupled cross-regression was proposed for building the coupled mappings between LR and HR feature space based on the advantages in the power ability of kernel technique to describe nonlinear space and the low computational complexity produced by spectral regression theory, which is a creative point in this study. Meanwhile, cross-regression was developed to build the coupled mappings between the low-dimensional embeddings of HR/LR samples and themselves. The proposed algorithm not noly utilize the relationships of HR samples but also ones of LR samples, thus effectively improve the representation ability of the unified feature space. Finally, the proposed CCR/KCCR and its improved algorithm ICCR/KICCR were comprehensively tested on two public databases named FERET and CMU PIE with the cases of single kernel and compound kernel. Experimental results showed that the proposed algorithm had a great improvement on the very low resolution such as8×8in recognition performance with the average improvement of6%, in comparison with the existing related algorithms based on coupled mappings model such as CLPMs, CLDMs, KECLPMs. Moreover, the computational complexity of the system was declined from cubic level to square level and even linear level. Thus, the speed of the system was greatly enhanced.(4) A new recognition-oriented face super-resolution algorithm was proposed, namely a resolution enhancement algorithm based on tensor eigen-transformation (TET) with the consideration of the acquisition of HR face image at-a-distance, which aims to improve the inconsistency between enhancement and recognition. The proposed algorithm includes two modules:resolution enhancement-oriented face detection at-a-distance and resolution enhancement based on tensor eigen-transformation. The proposed detection module combines the improved skin models such as H-SV and C’bC’r color spaces and the improved AdaBoost algorithm to realize face detection at-a-distance with pose and illumination variation cases, thus provide the input face samples for the next module. The proposed resolution enhancement module uses tensor analysis to perform eigen-transformation based on the feature vectors of singular value decomposition. It would build the combining weight distribution of the input LR face samples in the HR/LR training samples, thus achieve the global enhanced face images. Face residue compensation was then used for further processing the enhanced face images. Resolution enhancement with frontal view and good illumination cases was finally achieved, thus being beneficial to improve the recognition performance. In addition, the proposed three algorithms in this dissertation (FisherNPE, KCCR, TET) and two representative algorithms (CLPMs and S2R2) were comprehensively tested on indoor environment condition. Experimental results demonstrated that the proposed three algorithms achieved relatively better performance under the various combinations of distance (resolution), illumination and alignment.