Facial Action Unit Detection And Micro-expression Analysis

In recent years,with the development of artificial intelligence,facial expression analysis plays an increasingly important role in the fields of human-computer interaction and medical treatment.Basic expression(happiness,sadness,surprise,fear,anger and disgust)analysis technology is becoming more and more mature,but there are still some problems.First,basic expressions are difficult to express people's rich emotions.Second,basic expressions can be forged and suppressed,and it is difficult to reflect people's true feelings.The facial action unit defines the muscle activity of different face regions,and the combination of the facial action unit can express richer emotions.Furthermore,the micro expressions as a spontaneous expression can reflect people's true emotions.This paper mainly studies the facial action unit detection and facial micro-expression analysis.The main research work is as follows:1.In the feature extraction of facial action unit,the advantages and disadvantages of traditional methods and convolutional neural networks are compared and analyzed.The classical convolutional neural network under transfer learning is used to complete the feature extraction of facial action units,and the experimental results show its good performance.2.This paper uses the restricted Boltzmann machine to establish the facial action unit distribution,explores the relationship between the facial action units,and establishes the connection between unlabeled images and labeled images.Through adding unlabeled images to the training of the model,the semi-supervised facial action unit detection is completed.3.The structure of convolutional neural network with recurrent neural network is proposed to make use of the time series information in the facial action unit in this paper.The convolutional neural network extracts the spatial features of each image,and the recurrent neural network extracts the temporal features between the images,so as to realize the facial action unit detection for multiple images at the same time.4.Micro-expression has two characteristics:low intensity and short duration.For micro-expression recognition,this paper divides the facial area according to the muscle structure of the face,obtains facial feature points that can express the main expression changes through dense sampling,and further utilizes optical flow method to quantify the variation of the feature points in each area.Combining the above three points,we propose the Dense Sampling Optical-flow's Mean Magnitude and Angle(DS-OMMA)feature.Experimental results demonstrate that we achieve the best recognition accuracy on the two main datasets of the micro-expression so far.Moreover,we verify that the features proposed in this paper are better to describe the characteristics of different micro-expressions through feature visualization.