Research On Facial Expression Recognition Algorithms Based On Manifold Learning

Facial expression recognition is the basis of understanding emotions. It is an important topic in intelligent human-computer interaction to be solved, and is also an effective way to explore and understand intelligence. Due to the diversity and nonlinear characteristic of facial expression itself, the utilization of linear feature extraction approach is hard to obtain satisfactory recognition results. This thesis mainly researches feature extraction from facial expression images with the graph embedding based manifold learning techniques, aiming to overcome the recognition problem under the insufficient amount of (labeled) training samples. The innovative work of this thesis includes:1. Nearest feature line (NFL) classifier is utilized as a similarity metric and first introduced in manifold learning. Based on this metric we propose fuzzy local nearest feature line algorithm (FLNFL), where a graph based on feature line (straight line passing through each pair of feature points belonging to the same class) is established, in which the vertex is the linear combination of feature samples rather than isolated feature point. By this way, it can approximate variants of the two prototypes it connects, and virtually provides an infinite number of feature points of the class. The generalization ability of this class is thus increased. In order to differentiate overlapping samples, moreover, such as blended expressions, fuzzy assignment technique is adopted to specify the degree of membership in a set, rather than just the binary is or isn’t a member. Experimental results suggest that FLNFL algorithm provides a better representation of expression features and yield promising performance in facial expression recognition. Especially on smaller scale dataset such as JAFFE database, the advantage of FLNFL is more significant.2. As a newly proposed classifier, the benefits and limitations of shortest feature line segment (SFLS) are firstly discussed. A novel line-based similarity measurement combining NFL and SFLS named the enhanced SFLS metric is then proposed, which calculates the distance from point to line preceded by considering the angle between point and the vertexes of the line. As a result, the disadvantage of both NFL and SFLS can be avoided. Based on this metric, a linear discriminant function is defined, by analogy to the optimization concept of LDA. After being embedded into a low-dimensional subspace, data points of the same class cluster closer, whereas neighboring points from different classes keep away from one another. Experimental results on two widely used facial expression databases confirm the effectiveness of our proposed method.3. In traditional graph-based semi-supervised manifold learning methods, discriminant analysis is carried out over labeled data, while unlabeled data are exploited to capture the geometric structure, which has little effect on discovering the discriminative structure of data points. The class information of large amounts of unlabeled data, which may be helpful to uncover class distribution, is ignored in decision making. Moreover, they cannot promise good performance when the size of the labeled data set is small, as a result of inaccurate class matrix variance estimated by insufficient labeled training data. To address the above problem, a novel method of semi-supervised dimensionality reduction, called probabilistic semi-supervised discriminant analysis (PSDA), which first explores the statistical distribution information of unlabeled instances, and then integrates the additional information with ground-truth class information of labeled data to search for the most discriminative features of data points. Experimental results testify that the soft assignment technique is appropriate to express the inherent uncertainties of data points caused by environmental variations, such as illumination conditions or perspectives.4. A graph-based semi-supervised learning algorithm based on PSDA, termed as graph-embedded probability-based semi-supervised discriminant analysis (GPSDA) is developed. By introducing a similarity measurement of fuzzy sets to investigate the inexact class information of unlabeled data, an adjacency graph is modeled based on both neighborhood structure and category information, which is more relevant to classification compared with unsupervised graph constructed in traditional graph-based semi-supervised dimensionality reduction technique. Since more information is learnt from unlabeled data, GPSDA is expected to enhance performance in classification task. Experimental evidence on face and facial expression recognition suggesting that GPSDA is more effective compared with other related algorithms. It also shows certain robustness to the insufficient training samples recognition problems.