Research On Domain Adaptation For Facial Expression Recognition Based On Generative Adversarial Network

Emotions play an important role in people's daily life.Rich emotions are helpful for speakers to express their thoughts.It is one of the important goals of humancomputer interaction to develop an automatic facial expression recognition system for natural scenes.But in fact,facial expression images in natural environments often contain a variety of face angles,background noise,uneven illumination,and even partial occlusion.At present,facial expression recognition is mostly concentrated in the laboratory environment.There are artificial controlled gestures and illumination,so it is convenient to collect such data.On the contrary,there is a lack of a large number of labeled samples in the natural facial expression data sets.The cost of labeling large amounts of data is extremely expensive,which makes it possible for us to build effective algorithms to execute effectively in the case of little or no labeled data.Usually,other data such as data sets in the laboratory environment are used for transfer learning.Therefore,how to use the large data samples in the laboratory environment to help the training of data samples in the natural environment has become a brand-new lesson.In recent years,the generative adversarial network has been widely used because of its good generation effect.Therefore,two domain adaptation methods are proposed by using the generative adversarial networks in this paper.The main contents of this paper are listed as follows:1)Label-guided domain adaptive method in generative adversarial networks is proposed.By introducing the condition of emotional label,the method can generate facial expression images close to the real natural environment which meet the corresponding condition of emotional label,and expand the facial expression database of the natural environment.The classifier can learn the emotion-related features of laboratory environment and the emotion-related features of natural environment at the same time,so as to improve the recognition rate of facial expressions.It is not necessary to train the classifier to perform the task of facial expression recognition after generating all images.Experiments show that the average recognition rate of the seven expressions on the natural scene database RAF-DB is increased by 3%.2)A conditional adversarial domain adaptation method based on generative adversarial networks is proposed.This method embeds adversarial learning in deep network to represent learning separation and transferable domain adaptation.This method uses the recognition information transmitted in classifier prediction to assist adversarial adaptation.The key of the model is to capture cross-covariance between feature representation and classification prediction in source and target domains by using multi-linear mapping,so as to improve the discriminant ability of domain discriminator.Entropy conditioning is used to control the uncertainty of classifier prediction to guarantee transferability.The average results of cross-database experiments on the laboratory environment database BU₃DFE and natural scene database RAF-DB reached 71.3%,which was 44% higher than other mainstream domain adaptation methods.3)We design and implement a prototype system of domain-adapted expression recognition based on generative adversarial network.The operation interface of the prototype system is designed with MATLAB,and the core algorithm of the system is realized with Python,Pytorch and Numpy.The prototype system includes two modules: image generation and classification test.The implementation of the prototype system verifies the availability and validity of the proposed methods.