Static Facial Expression Recognition And Estimation In Natural Environment

With the rapid growth of computing ability power and the accumulation of image data,facial expression recognition,especially based on natural environment,has gradually become one of the hot research areas of artificial intelligence.At present,the facial expression recognition technology in the laboratory environment is relatively mature.However,facial expression recognition in natural environment is closer to the real application,and the effect of the related algorithm still needs to be improved.There are two representation models of facial expressions,discrete and continuous dimensions,and the two representation models both have great application prospects in education,medical care and security.Therefore,this paper focuses on facial expression recognition(discrete)and estimation(continuous dimension)based on natural environment scenarios.The main work of this paper is as follows:(1)Facial expression recognition in natural environment is more challenging than that in the laboratory environment.The images in the real world environment have large variations in non-expression factors such as pose,background,illumination and so on.Besides,there are some samples with low valence,which are prone to confusion in the feature space.Based on the above points,a facial expression recognition method based on combined valence-sensitive loss and center loss is proposed.Valence-sensitive loss is used to solve the confusion caused by low valence.While the introduction of center loss is used to mitigate the effects of non-expression factors,so that similar samples could be gathered together in the feature space.Finally,the convolutional neural networks with two-branch structure are designed to optimize the two loss functions and the softmax loss to realize multi-class facial expressions recognition.(2)Both discrete and continuous dimensional representation models have great application prospects.Different from the category of facial expressions,some application scenarios want to obtain the state value of the facial expression.Therefore,this paper also studies the facial expression estimation under the continuous dimension(valence-arousal model).A facial expression estimation algorithm based on two-level attention mechanism with two-stage multi-task learning is proposed.To overlook the various non-expression factors in the real scene,the two-level attention is proposed to automatically extract the position-level feature with the residual attention block,and adaptively extract the layer-level using the Bi-RNN(Bi-directional Recurrent Neural Network)model based on self-attention to improve the ability of neural networks to extract features.Secondly,a two-stage multi-task learning structure is designed.On the one hand,the feature representations of discrete models and continuous dimensional models are jointly studied to enhance the precesion of feature representation under continuous dimensional models.On the other hand,multi-task learning uses the correlation of valence and arousal to predict the values of both.In addition,we use the Tukey's biweight loss function to alleviate the influence of inconsistent or erroneous samples.Finally,extensive experiments in the AffectNet dataset demonstrate the effectiveness and generalization of the proposed facial expression estimation algorithm.