Posed Vs Spontaneous Expression Analysis And Recognition Research

Automatic facial expression recognition has been an active research area in recent years due to its wide applications in human-computer interaction, security and health-care, etc. Generally, two kinds of features are used for detecting facial expressions: geometric and appearance features. In order to extract geometric features, facial fea-ture points should be detected or tracked accurately. Although considerable progress has been made in the field of facial feature point detection and tracking, accurate fea-ture point tracking is still very challenging. Besides, it’s valuable to recognize posed vs. spontaneous expressions which are part of expression recognition. Most of them only consider one specific expression, such as smile or pain. Little research takes all six basic expressions into account to distinguish posed and spontaneous expressions. Moreover, no report consider gender and six basic expression as privileged informa-tion to recognize posed vs. spontaneous expression. Therefore, this thesis presents the corresponding method aiming at above problems. The details are as follows:(1) We propose an active feature point labeling method. First, the spatial rela-tions among feature points are modeled by a Bayesian Network. Second, the mutual information between a feature point and the remaining feature points is calculated in two steps:in the first step, to identify the most informative facial region, the mutual information between one facial sub-region and the other sub-regions is calculated; in the second step, the mutual information between one feature point and the other feature points in the most informative facial sub-region is established to rank the facial feature points. Users provide labels of the feature points according to their mutual information in descending order. After that, the human corrections and the image measurements are integrated by the Bayesian Network to produce the refined annotations. Simulative ex-periments on the extended Cohn-Kanade (CK+) database demonstrate the effectiveness of our approach.(2) We propose comprehensive analyses of the differences between posed and spontaneous expressions from visible images. First, geometric and appearance fea-tures are extracted from the difference images between apex and onset facial images. Secondly, the differences between the posed and spontaneous facial expressions are an-alyzed through hypothetical testing methods from three aspects:on overall samples, on samples with different genders, and on samples with different expressions. Thirdly, Bayesian networks (BNs) are used to classify posed versus spontaneous expressions from the same three aspects. Statistical analyses on the NVIE database demonstrate the importance of the geometric and appearance features when discriminating posed and spontaneous expressions. Gender effect exists on the differences between posed and spontaneous expressions. It is easier to distinguish posed happiness from sponta-neous happiness than other expressions. Recognition experimental results confirm the observations of statistical analyses in most cases.(3) We propose a new method to recognize posed and spontaneous expression through modeling their spatial patterns. Gender and expression categories are employed as privileged information to further improve the recognition. The proposed approach includes three steps. First, geometric features about facial shape and Action Unit varia-tions are extracted from the differences between apex and onset facial images to capture the spatial facial variation. Second, statistical hypothesis testings are conducted to ex-plore the differences between posed and spontaneous expressions using the defined ge-ometric features from three aspects:all samples, samples given the gender information, and samples given expression categories. Third, several Bayesian networks are built to capture posed and spontaneous spatial facial patterns respectively given gender and expression categories. The statistical analysis results on the USTC-NVIE and SPOS databases both demonstrate the effectiveness of the proposed geometric features. The recognition results on the USTC-NVIE database indicate that the privileged information of gender and expression can help modeling the spatial patterns caused by posed and spontaneous expressions. The recognition results on both databases outperform those of the state of the art.(4) We propose a Bayesian network approach for learning with privileged informa-tion and apply the method to posed vs spontaneous expression recognition. We propose to incorporate the privileged information through a3-node Bayesian network. We fur-ther mathematically evaluate different topologies of the3-node Bayesian network and identify those structures, through which the privileged information can benefit the clas-sification. We conduct experiments to recognize posed vs spontaneous expression using gender as privileged information on USTC-NVIE database. Experimental results show that triangular and V structure Bayesian network are helpful to improve the recognition when using gender as privileged information. It indicates that gender promotes posed versus spontaneous expression recognition.