Facial Expression Recognition Using Geometric Features And Appearance Features

Facial expression recognition is an important research field in recent years, Facial expression can reflect a person’s emotion state, which plays a crucial role in the social communication. Now the facial expression has been widely used in affect analysis, intelligent learning and human-computer interaction. Many researchers have made efforts in the expression recognition and achieved good success in the field, however, the face appearance differences, as well as makeups make the change of expression become both subtle and complex. Hence, facial expression recognition is challenging. As for the automatic facial expression recognition, how to extract the effective features is one of the key factors to achieve a high recognition rate.A novel method using hybrid geometric and appearance features of the difference between the neutral and fully expressive facial expression images is proposed for facial expression recognition in this thesis. Base on the relevant work of the existing solutions on feature extraction, this study has made the contributions including:(1) Using the different changes between expression image and neutral expression under the normalized conditions as the feature, in order to weaken the impact of appearance differences on facial expression recognition. Because of the different appearance among people, there is a big individual difference in the common part of facial expression. The difference between expression image and the neutral of each one can tends to emphasize the facial parts that are changed from the neutral to expressive face in general, and eliminate in that way the identity of the facial image.(2) Active Shape Model (ASM) is improved to accurately detect face and acquire important facial component regions. By detecting the geometry location of the eyes and mouth automatically, then a good initialization condition for locating the key feature point in ASM is provided. Thus the information of geometry feature which is extracted by the improved ASM can describe the geometric feature of the face more accurately.(3) The facial feature points are extracted using the extended Active Shape Model (ASM), then the geometric features are obtained based on the point displacements between the normalized neutral and expressive facial expression images. The texture features are obtained based on the relations between gradient vectors from the normalized neutral and expressive facial expression images. The hybrid features include facial feature point displacements and local texture differences between the normalized neutral and expressive facial expression images.In this thesis, the difference ASM features describe the facial shape difference between neutral and expressive faces, while the local texture features represent the appearance in detail. Experiments with images from the extended Cohn-Kanade (CK+) facial expression database have validated the proposed features, and achieved good recognition performance with a Support Vector Machine (SVM) classification method.