Facial Expression And Action Unit Recognition Based On Prior Knowledge

Studies show that facial expression is important for human communication, there-fore, facial expression recognition is a key topic in anthropomorphic human-computer interaction research. Begin from the1990s, automatic expression recognition research is increasingly active, many research institutions have paid attention to the study, and obtained some achievements. But in general, it is still in the research stage. Expression recognition can be divided into two types:action unit recognition and expression clas-sification. The general framework extracts various geometric and appearance features firstly and then models classifier using machine learning approach to recognize facial action unit and expression. This feature-driven method is vulnerable to imaging condi-tions (such as illumination and camera angles), and it is difficult to expand. While the facial muscles’temporal relationship (i.e., the facial prior knowledge) remains. So, as useful and necessary supplement information, facial prior knowledge may benefit the expression recognition. Therefore, this paper conducts a preliminary exploration on facial expression and action unit recognition using prior knowledge, as follows:(1) propose to recognize expression using facial action unit. This method models the probabilistic relationship among the facial action units and expressions as the prior knowledge of expression recognition. Firstly, determine the most important action units for each expression. Secondly, model Bayesian network to capture the relationship among expression and its relevant AUs. Lastly, combine with traditional classification algorithm to recognize expression.(2) propose to recognize AU on incomplete annotation data using expression. This method models the relationship among expression and AU to be the prior knowledge. Firstly, structural expectation maximization algorithm is used to learn the probability graph model among the expression and AUs from the incomplete data, and then com-bining with general classification algorithm to simultaneously detect all of the AUs.(3) propose early expression recognition using Hidden Markov Models (HMM) capturing the temporal information. Firstly, construct HMM for each expression and determine the entropy threshold on the validation set. During testing, the facial expres-sion segment is input to the model by frame to get the probabilities of each expression category and compute the entropy, and then add frames to reduce the entropy until the entropy threshold or the segment over. Then determine the expression category accord-ing to the probabilities.