Research On Robust Facial Expression Recognition

Facial expression is an important way of communicating emotional informationand harmoning human relationship. Through facial expression recognition, we obtaina large amount of valuable information. Facial expression recognition, one approachof human visual perception, using machine learning to classify facial expressionfeatures, has become an important research topic in the field of multimediainformation processing, human-computer interaction, affective computing,intelligent control, machine vision, image processing, pattern recognition, and so on.There are three major challenges in recognizing facial expressions: firstly, there areindividual differences in the same facial expression, that is, people with differentethnics, gender and cultural backgrounds have different features; secondly, theproblem that the training and testing data are in the different feature space and havethe different distribution, which is not considered in the semi-supervised facialexpression recognition algorithms; finally, face occlusion is very common in thesystem of facial expression recognition. These problems cause the decrease ofrecognition rate and robustness. In order to improve the classification accuracy androbustness of facial expression recognition algorithms under complex environmentconditions, and to break the limitation of facial expression recognition on specialcondition, this thesis gives an in-deep investigation on the robust facial expressionrecognition algorithms, and the major techniques and contributions are listed asfollows:1. To address the low recognition rate problem caused by the individualdifferences and appearance similarities in facial expressions, two novel algorithmsbased on discriminative component analysis are proposed, i.e, local discriminativecomponent analysis (LDCA) and two-dimensional local discriminative componentanalysis (2D-LDCA). LDCA is a vector-based approach, and2D-LDCA is amatrix-based approach. A set of nearest neighbors of a testing sample from trainingset are determined to capture the local data structure. Therefore, the proposedalgorithms can extract the most efficient features for facial expression recognition,and obtain higher classification accuracy. In the experiments for two-classclassification, the recognition rates of LDCA and2D-LDCA between anger anddisgust, fear and surprise, fear and sad expression groups are above90%. In theexperiments for multi-class classification, the recognition rates of LDCA and 2D-LDCA are above90%and80%on the databases which contain different ethnicsand genders. Compared with baseline methods, our proposed methods have theobvious superiority.2. Semi-Supervised Learning assumes that the training and testing data are inthe same feature space and have the same distribution. However, these individualdifferences and head rotations in facial expressions often lead to false assumptions,which in turn can decrease the success recognition rate. A novel algorithm calledsemi-supervised learning adaptive boosting (SSL-AdaBoost) is proposed to solve theproblem. Our proposed method has employed a fashion of knowledge transfer toconstruct a high-quality classification model, and to some extent, overcome problemof the training and testing data being drawn from different feature distribution.Under semi-supervised learning framework, we recognize the facial expressionimages from multiple databases and multi-pose data respectively. Compared with thelabel propagation method and many other semi-supervised learning methods,simulation experiments show that the proposed algorithm achieves the highestclassification accuracy. The recognition rates can be up to73.33%on multipledatabases and87.71%on multi-pose data, respectively.3. For the problems of information missing and noise interference, which arecaused by partial occlusion of the face, a facial expression recognition algorithmbased on Exact Augmented Lagrange Multiplier (EALM) and SSL-AdaBoost isproposed. Firstly, EALM is used to recover the occluded facial expression regions.EALM can exactly recover the occluded images, even if the occlusions of faceimages are significantly corrupted. Secondly, linear discriminant analysis is appliedto extract facial expression features. Finally, SSL-AdaBoost is used for recognizingthe partially occluded facial expression images from multiple databases.Experimental results show that our algorithm can achieve higher recognition ratesthan the baseline algorithms under the condition of random occlusion.