Research Of Facial Expression Based On Multi-level Fusion

Facial expression is able to contain the information which cannot be containedby language. It is a very important way to transfer information in the interpersonalcommunication. Researching the method of facial expression recognition helpscomputer to infer people’s mental state and communicate with human naturallyaccording to facial information. So, a valid method has the application value. What’smore, the researching includes image procession, pattern recognition and machinelearning. It is a field with powerful intersectionality. So, facial expressionrecognition is one of the most important fields for visual information computing.However, due to the impact of individual difference, facial expression recognition isstill a difficult problem.This paper applies the fusion technology to facial expression recognition, andexploring how to implement an integration which fuses expression classifier withvarious expression recognition performance and different features with ability ofcharacterizing the expression by fusing multi-layered identification system. Improvethe ability of recognition algorithm to cognize natural human expressions throughthe complementarity performance. For the fusion of multiple classifiers and features,in this paper, not only the multi-classifier fusion were discussed, but also thehierarchy fusion of multi-feature and multi-classifier were studied, which are allbased on stacking algorithm, and the research results can apply into the human facialexpression recognition successfully. The followings are the work in this paper:(1) Extract multiple facial expression features. Several good features areproposed in the existing research performance. However, there is no any featurewhich was proved better than others feature to descript facial expression. In thispaper, four feature with different principles are used to extract facial expression andthey are the base for all the research.(2) Considering the difference between classifiers and the performance of fusingmulti-classifier, the paper proposes the multi-classifier fusion expression recognitionmethod based on stacking, which contains the base-level and the meta-level. Thefusion method achieves the classification of the results of base-level classifiers byusing meta-level classifier. Analyzing the samples’ different contribution, we canestablish the multi-classifier fusion model, which can help us improve the classification ability. It is good for increasing their complementarity to enhance thedifference between base-level classifiers, which is benefit to improve the cognitionability of fusion method. Therefore, in the paper, kappa-error diagram is used toselect the base-level classifiers. The pair-classifiers’ error rate and diversity are bothimportant for fusion performance, which was used to determine the base classifiersin kappa-error diagram.(3) Action unit, with the character of being independent from individual faceshape, can be used to identify the basic block action. And due to that character, whenwe construct the facial expression feature based on AU, it can eliminate thedifference between various faces and describe the intrinsic character of expressioneffectively. This paper constructs the facial expression vector based on theprobability certain of specified AU in facial expression image, which can improvethe recognize rate obviously. Combining with stacking fusion arithmetic, proposesthe theory, a multi-level expression recognition method based on AU description.With the forecast probability we get from the base-level, we can form AU charactervector by combining the prediction of AU, and get the final expression through themeta-level classifier.Based on multiple performance-judgement standards, we widely conduct aboutthese two fusion methods respectively. The result shows that, compared with singleclassifier and voting fusion method, multi-classifier and multi-feature facialexpression recognition method, based on stacking algorithm, has better recognitioneffect; Compared with mainstream algorithm of expression recognition, itsrecognition performance improves a lot.