A Study Of Automatic Facial Expression Analysis And Recognition System

The face is a rich source of information about human behavior. A facial expression results from one or more motions or positions of the muscles of the face. They are closely associated with our emotions. Facial displays indicate emotion and pain, regulate social behavior, and reveal brain function and pathology. Automatic facial expression analysis and recognition plays a valuable role in society psychology and other relative research fields. It is quite difficult to analyze and recognize facial expression because the face is characteristically non-rigid, and there exist differences among individualities and races of the individuals to be researched. Based on the correlative research at home and abroad, this dissertation discussed some problems related to automatic facial expression analysis and recognition. Research in this dissertation includes the recognition of six basic expressions from static images with mixed facial expressions, and the analysis of dynamic image sequence according to facial action coding system. The main contents of this dissertation are as following: 1) Semi-automatic facial feature points marking method is proposed to decrease the tedious work in marking facial feature points of facial image with hand in the process of defining training sets of active appearance models (AAM). This method proposed corresponding semi-automatic method for different facial organs according to their features, such as corners, edges. 2) The ability of representing facial expression of AAM facial expression feature is analyzed by using spearman rank correlation analysis and non-metric multidimensional scaling method. According to the results of multi-variable statistical analysis, a multi-layer perceptron artificial neural network is constructed to classify the AAM facial expression feature. The experiments show that the performance of this method is better than other similar systems. 3) To recognize facial action units in face image sequence, three kinds of facial expression feature are extracted: facial geometrical features and transient features, difference image features and its gabor-ICA(Independent Component Analysis) feature. To exclude the correlation between these features, inverse compositional image alignment method is used to extract difference image. 4) As to the high dimension and redundancy of difference image feature in AU (Action Units) recognition, a feature selection algorithm is applied. The feature selection algorithm based on Margin is improved in this dissertation. Conjugate gradient optimization method is used to speed the convergence of the performance function in the original method, and the difference evolution method is used to select the good initial position of the optimization process. Finally, multiple classifiers are designed to recognize three different kinds of facial expression features. The experiments show that the combination of multiple classifiers has good results compared with every single classifier.