Facial Expression Recognition Based On Rough Set

The study of human centered emotion and cognition, including affective computing and emotion recognition, is a hot research topic in artificial intelligence. Achievements on these research topics will push the development of human-computer intelligent interaction (HCII). Although there are already some achievements of affective computing and emotion recognition in recent years, there are still some key problems unsolved due to the absence of solid theory basis of psychology and cognition. In this thesis, feature selection methods and emotion recognition methods in a facial emotion recognition system are studied. These research works can not only push the development of emotion recognition and emotion simulation, but also are important for the applications in HCII, E-learning, medical caring, game, etc.In this thesis, attribute reduction algorithms are studied based on rough set theory and used for feature selection of a facial emotion recognition system. Geometrical face features are taken as the research objects since it is intuitionistic. Important features for emotion recognition are studied based on attribute reduction algorithms. Furthermore, efficient emotion recognition methods are also studied based on rough set theory and ensemble learning. The major achievements of this thesis are as follows.1) A feature selection method for emotion recognition based on rough set theory is proposed.In this thesis, the attribute reduction algorithms based on rough set theory are introduced and proposed for feature selection methods for emotion recognition. Based on traditional rough set theory, the attribute reduction algorithm based on conditional entropy and the attribute reduction algorithm based on feature selection are used as feature selection methods for emotion recognition. Meanwhile, a novel emotion recognition method based on RS plus SVM is proposed. The experiment results show that the attribute reduction algorithms can reduce the feature dimension and extract the important features for emotion recognition. Based on these features, better recognition result than traditional feature selection methods can be achieved.2) A self-learning expression feature selection algorithm is proposed.In this thesis, based on the idea of domain-oriented data-driven data mining theory, an attribute reduction algorithm for continuously valued information systems is studied, and a self-learning attribute reduction algorithm is proposed. Discernable-ability of conditional attribute set with respect to decision attribute set is taken as an important property of knowledge, and a measurement of discernable-ability is proposed based on rough set theory. According to the criterion that discernable-ability should hold in the course of knowledge acquisition, a self-learning attribute reduction algorithm (SARA) is proposed for continuously valued information systems. Its parameter can be obtained automatically from training data set. Experiment results show that the method can get good result even if there is no prior domain knowledge. SARA is taken as a feature selection method for emotion recognition in this thesis, and good recognition result is achieved based on the features it found. Furthermore, the features which concerned mouth are found as the most important expression features.3) A selective ensemble feature selection method for emotion recognition is proposed based on rough set theory and ensemble learning.At first, an algorithm based on discernibility matrix is adopted and multiple reductions are generated, correspondingly multiple candidate classifiers are trained. Secondly, the double fault method is taken as the measure of the diversity of the candidate classifiers, and the candidate classifiers are clustered according to measurement of the double fault method, and the most diverse classifiers are selected from each pair of clusters. At last, the criterion of relative majority voting is adopted for the selected classifiers, and the output of the ensemble is gotten. Experiment results show that good recognition result can be achieved based on the proposed method.4) A dynamic ensemble feature selection method for emotion recognition is proposed based on extended rough set theory and ensemble learning. At first, an algorithm for calculating the core of a decision table is proposed based on the domain-oriented data-driven data mining theory. Secondly, an algorithm for finding multiple reductions based on conditional entropy is proposed. Accordingly, multiple candidate classifies are generated using this algorithm. At last, a dynamical selective measure is used to select the most suitable classifier for each unseen sample according to local property of the candidate classifiers, and recognition result is gotten accordingly. Experiment results show that good recognition result can be achieved based on the proposed method.5) An audio and visual double module emotion recognition system is developed.A double module emotion recognition system based on both audio and visual information is developed. It can recognize facial expression, emotional speech and expression of the combination of facial and speech emotions in time. Good recognition results are achieved in real testing environments.