Research On Multi-layer Fusion Expression Recognition Method Based On Action Units

In human life and communication, facial expression is one of the most important methods to convey human being emotions. Using computers to learn and infer human expression changes has become more and more popular in the artificial intelligence field. As we seek to design computers that related better to people and can deal with the rich and subjective nature of both verbal and non-verbal communication. However, in reality environments, facial expression is diverse and complicate, coupled with the impact of different race, color, gender, and age, which makes the research of facial expression recognition become into a huge challenge. Therefore, to overcome this hard stone, we need to find and design a diverse facial expression recognition model.Among the existing approaches, the Action Units(AU) based facial expression recognition is one of the effective method. However, finding an accurate AU recognizer is still a big problem, for no facial features or classifiers can be proved always perform well. In order to improve the overall performance, our paper proposes to combine the complementary contribution of multiple features and multiple classifiers in AU recognition with stacking architecture. We verify the complementarity between multiple features and multiple classifiers with expression recognition, and analysis the AU relationship of expression recognition by association rules. The main research work and innovations are as follows:1. Providing a kind of expression recognition algorithm based on AU fusion. To eliminate human individual differences, this paper uses FACS technology to divide face images into AU blocks, employing four kinds of feature extraction methods and three kinds of classifiers to recognize AU, and establish a stacking framework to recognize facial expressions based on AU probability.2. Verifying the complementary of different feature and classifiers on facial expression recognition. This paper employs Kappa-error graphical analysis method to analyze the fusion performance of typical features combination and classifiers combination. Especially, this paper designs three compared experiment between entire face and AU blocks, single feature and multi-feature combination, single classifier and multiple-classifiers fusion of facial expression recognition. Experiments show that the combination of multi-features and multi-classifiers fusion method based on AU can overcome the single feature or classifier itself, and with a strong complementary on facial expression recognition.3. Establishing an expression recognition model of weighted AU fusion based on association rules mining. This paper selects geometry feature to extract AU blocks, and uses classifiers to predict expression label of each AU. Employing an Apriori algorithm to find a strong relationship between AU predicted expressions and target expression, which is an idea to calculate AU weight. Finally, through the weight matrix and probabilities of AU, an expression recognition model is established. Extensive experiments show that our proposed algorithm achieves a high recognition effect and has obvious advantages among others.