Research On Spontaneous Facial Expression Recognition

Facial expression is one of the most important ways for people to communicate their feelings with each other. Facial expression recognition has been one of the key issues in the research of personification human-computer interaction. Presently, there are many research institutions and colleges, both at home and abroad, who have proposed many methods to solve this problem, and some progress has been made. However, expression recognition is still in the lab stag, and most researches are based on the posed expression. The appearance of posed expression is stiff, rigid, and different from human spontaneous expression. Therefore, researches based on the spontaneous expression are propitious for the reality application of expression recognition.For the characteristic of spontaneous expression, this thesis proposes three methods for expression recognition and applies one of them to video emotional semantic implicit tagging. The detailed is as follows.(1) We propose a spontaneous facial expression recognition method based on the head motion. In the method, firstly pupils' coordinates are detected by eye location in the onset and apex frame. Secondly, 30 appearance features are extracted from the apex frame by AAM. Finally, SMO is employed for classification. The experimental result indicates that head motion features are good at discriminate fear; they could classify expression by themselves or be added in other feature set to improve the accuracy.(2) We propose a method of attitude recognition based on the head motion. In the method, pupils' locations are firstly detected in each frame. Then between-eyes displacement in successive frames is calculated. If displacement in x-axis is more than that in y-axis, then result of these two is positive, else it is negative. Finally, the voting algorithm is employed to get the final result. The experimental result indicates that this method could detect user attitude in real time.(3) Propose a recognition method based on the feature point tracking. In the method, firstly all expression sequences are normalized according to their pupils' coordinates. Secondly, 23 points are labeled manually in the onset and apex frames. Then Kalman filter is used for tracking. Two kinds of feature (point displacement feature and points distance variation feature) are calculated. Finally, Hidden Markov Model is employed as classifier. The experiment result indicates that Kalman Filter point tracking method could detect the right place of point. However, the classification result based on the point displacement feature is not as good as that based on the points distance variation one and the accuracy based on the posed expression is better than that based on the spontaneous one.(4) We propose a video emotional semantic implicit tagging method based on spontaneous expression recognition. The first expression recognition method is used to recognize expression and emotional semantic tagging is inferred from the recognition result. The result shows that if the expression recognition method is reliable, emotion could be inferred from it and this implicit tagging is feasible.