Facial Expression Recognition Based On The Hybrid Feature And Neural Network Ensemble

Facial expression plays an important role in daily life, it is a main way of human nonverbal communication, and it is an important supplement of exchanged language. Facial expression recognition is the basis of emotional understanding, and premise of computer understanding or expressing emotion of human. Recently, with the development of affective computing, facial expression recognition is become a very active research in scientific community.Facial expression recognition is a cross-subject in the fields of pattern recognition, machine vision, physiology and psychology. Because of the complexity and specificity of facial expression, which make facial expression recognition become one of the most challenging problems and have a broad application prospect.Generally, facial expression recognition system includes the following three parts: expression image preprocessing, expression feature extraction and classification. As feature extraction and expression classification are the key steps of facial expression, this paper has a deep research on these two aspects. The main work is as follows:(1) Expression image preprocessing.First, we need to locate human face and feature areas. Our study is based on JAFFE database. In order to locate human face, eyes and mouth accurately, AdaBoost is adopted in this paper. Experiment shows it not only can sure accurate locating, but can satisfy real-time system's requirement on time. On the basis of locating face expression images, gray of expression images is equalized. After equalization the details of the image get clearer, and the distribution of gray levels of histogram gets evener. It also overcome great difference between gray levels of the same expression due to illumination, and make sure that learning and testing images are in the same condition.(2) Expression feature extraction.Feature extraction is the most important steps which result in the classification rate of the facial expression. When facial expression appears, it must cause facial deformation which includes the information for the expression classification. Therefore, this paper will denote these deformation by extracting the geometry structural features which reflect the change of facial shape and the local statistic features which reflect the change of facial texture, and eliminate redundant features as possible, in other words, we just use the necessary facial features for expression recognition. In order to extract geometry structural features, the method of AAM (Active Appearance Model) is used to locate feature points in the facial images, and extract 12 features which reflect facial deformation. To avoid the loss of the information, we generate local statistic feature using co-occurrence matrix as the supplement, more vividly describe the relationship amount the pixel and the statistic feature, in this paper we select the most effective statistic features to denote expression based on Fisher criterion. Experiment shows that the hybrid features can conquer the interference factor of the individual feature and the sunshine.(3) Facial expression classification.Because of the diversity and complexity of facial expression, it's difficult to classify the facial expression by linear classification. In this paper, we bring neural network ensembles classification into expression recognition. Neural network ensembles classifier which we build includes 3 sub-networks, each sub-network bases on RBF network. We choose 137 images from JAFFE database as the training samples, and rest 60 images to test. For the following seven expressions:"Angry","Disgust","Fear","Happy","Neutral","Sad"and"Surprise", experiments show that the average recognition rate can reach to 85.53% and 88.16% using single RBF classification and Neural network ensembles classification, the classification effect of ensembles classifier is better than single RBF classifier, and it improves the generalization ability of classifier. In order to strengthen the reliability of the algorithm in this paper, we choose 48 images from JLUFE database, and we also achieve very well recognition effect.