Regression-based Estimation Of Facial Action Units Intensity By Multi-modal Features

Facial expression information plays an extremely important role in the daily communication of human beings.People catch the emotions and state of each other by analyzing their facial expressions.With the development of modern information technology,the research of machine learning,image processing,artificial intelligence and other fields have got greatly breakthrough.More and more software applications and life scenes are beginning to generate demand for automatic recognition of facial expressions.The current machine learning algorithm has a good result in the preliminary determination of facial expressions,but it's not enough to simply get facial emotions of happiness,sadness,surprise or anger.More detailed information from facial muscles and expression intensity is needed.Facial Action Coding System(FACS)defined Action Units(AU)for facial expression information.Previous algorithms including deep learning classification,clustering algorithm and sparse representation for estimating the intensity of action units didn't perform well,because of the less discriminative properity between two neighbouring intensity levels and the large noise within individual facial information.At the same time,the lack of high intensity AU samples is also one of the reasons to prevent these methods from working.To solve the above-mentioned problems,this paper explores the correlationship between the AU activations and the specific facial regions,and proposes a new feature extraction method based on regions and feature fusions in the phase of feature extraction.In this paper,we select the strongly correlated regions for each AU,then only extract the feature from these regions.At the same time,as a supplement,we examine the correlation between AUs through data analysis.In the intensity evaluating phase we propose a new method for AU intensity evaluation.It builds a two-class classification about high intensity AU samples and zero intensity AU samples by SVM,then obtains a classification hyperplane.According to the distance between the sample under test to the classification hyperplane,we judge the AU intensity by ordinal regression.The AU intensity evaluating algorithm not only catches the strong separability between strong AU expression and weak AU expression,but also takes into account the progressive correlation between different intensity of AU,simultaneously take advantages of classification and regression.Experimental results on the widely used DISFA,FERA2015 datasets show that our algorithm outperforms the state-of-the-art AU intensity estimation algorithms.