| At present,the accuracy of expression recognition in laboratory scenes(including facing the camera,no occlusion,good lighting and other conditions)is close to 100%.However,the laboratory scene conditions are too restrictive and far from enough to describe the complexity of facial expression recognition.Therefore,facial expression recognition in real scenes often encounters more difficulties and challenges.For example,in the expression recognition task,the following three problems often appear.The first is the occlusion of the expression,that is,the eyes,mouth and other parts of the face are sometimes blocked by items such as sunglasses and hair;the second is the face with a large posture,that is,person’s head turned from the front and the side face image is not visible under the camera position;Finally,the facial expression data in real scenes usually follows long-tailed distribution,that is,neutral and happy expressions occupy the vast majority of expression data,while sad,disgust,fear and anger etc.only occupy a very small part of the expression data.The first two issues affect the recognition accuracy of all expression categories,and also affect the recognition accuracy of expression recognition tasks in unrestricted situations(i.e.,real scenes).However,the problem of the long-tail distribution of expressions will affect the recognition accuracy of expression model algorithms for minority expressions.Specifically,the model is more inclined to correctly recognize the majority of expressions,while the recognition accuracy for minority expressions such as anger and disgust is low.This paper proposes three corresponding solutions to these three problems under large-scale expression recognition.The specific research work is as follows:(1)Aiming at the problem of occluded facial expression recognition,this paper proposes a lightweight model based on the attention mechanism that mixes global features and local features,referred to as the mixed feature layer.Through this module,different weights are given to occluded areas and non-occluded areas.This paper complete recognition of facial occlusion expressions,and comparative experiments with other models and experiments on occlusion expression datasets to prove the effectiveness of the model.(2)Aiming at the facial expression recognition of large posture,this article is based on 3D facial modeling technology to reconstruct facial models and perform facial rotation and rendering operations in 3D space.Finally,through the improved model algorithm based on GAN proposed in this paper,the frontalization of the face is realized,and the experimental research on expression recognition is completed using the frontalized face image.(3)Aiming at the problem that facial expressions follow long-tailed distribution,this paper obtains the coordinates of key points of the face in 2D space based on the 3D face reconstruction algorithm,locates the important sensory positions of the face.This paper also based on the idea of Cut Mix(a data enhancement method),the key pixels of the minority expressions are used as the foreground image,and the non-key pixels of the majority expressions are used as the background image.The image cutting operation is performed to complete the data enhancement work of the minority expressions.Through this method,this paper increases the sample size of minority class expressions and increases the opportunity for the model to learn minority class expressions.A series of experiments were conducted to demonstrate the effectiveness of this method.In this paper,a large number of experiments were carried out on the public datasets RAF-DB and Affect Net respectively.The experiments verified the effectiveness of the model proposed in this paper,which can well recognize facial expressions in real scenes.In RAF-DB and Affect Net datasets accuracies are 89.54% and 65.03%,respectively. |
