Facial Expression Recognition Research Based On Emotional Feature Disentanglement Learning

Facial expression recognition plays an important role in affective computing,which has been widely used in human-computer interaction,drivers' abnormality monitoring,pain estimation,online education and other fields.Human beings express their emotions in different ways.Even if transmitting the same emotion,their facial expressions are different causing by skin color,gender and other identity attributes.At present,the common used deep learning methods have recognized expression effectively through extracting semantic features of image,but due to the small scale of existing expression datasets,it is still hard to learn expression features separated from personal identity.This paper studies the facial expression recognition method based on emotional feature disentanglement learning to solve the entangled problem between expression features and identity features.The main work includes:(1)Aiming at the problem of few samples in facial expression datasets,a facial expression recognition method based on multi-task learning with feature space disentanglement is studied.The sample space is expanded indirectly with the implicit data enhancement ability of multi-task learning,and model's generalization is improved.Considering the influence of personal identity on facial expression recognition,the attribute disentanglement method is used to map facial expression attribute and identity attribute into different latent spaces,and the training of encoder is supervised by the task of facial expression image reconstruction,so that the disentanglement of facial expression feature and identity feature is achieved.(2)The expression recognition model based on de-expression residue learning is studied.The input expression is mapped to neutral expression through expression mapping model to learn the disentangled representation of facial expression features.De-expression residue images are acquired through expression mapping.Different base-classifiers are designed for de-expression residue images in different feature levels.The importance of features is automatically learned through the feature sense layer to reduce feature redundancy in the fully connected layer of the base-classifier.Finally,a multiple base-classifier integrated decision recognition model is constructed to recognize different de-expression residue and reduce the influence of personal identity on expression recognition.(3)The expression recognition method proposed in this paper is verified on the standard expression datasets.Through comparative experiments on CK+and RaFD,and visualization of feature distribution before and after disentanglement,it is verified that the proposed expression recognition method based on multi-task learning with feature space disentanglement can effectively disentangle expression features and identity features and weaken the influence of personal identity attributes on expression recognition.At the same time,extended experiments are designed to further verify the generalization performance and recognition ability of this method.The experimental results on Oulu-CASIA and RaFD show that the expression recognition method based on de-expression residue learning can make full use of the hidden expression information in de-expression residue to improve the recognition accuracy.The reliability of the integrated decision recognition model is further verified by ablation experiments as well.