Unsupervised Cross-view Facial Expression Image Generation And Recognition

With the popularization of human-computer interaction and the continuous success of deep learning technology in the field of computer vision,facial expression recognition based on deep neural network has become a hot research direction in affective computing.There are usually three prominent problems in the study of facial expression recognition.On the one hand,it is difficult to guarantee that all the faces captured by image acquisition equipment in practical application are face positive images,and it is inevitable that a large number of facial images with various postures will appear.However,most of the current facial expression recognition research is usually only focused on the frontal images,when the face pose has a large angle of rotation,the recognition accuracy is greatly reduced.It is still a very challenging task to reliably identify the facial expressions of different postures.Secondly,most of the current facial expression image databases are mainly composed of facial expression images of the frontal face,and the number of labeled multi-pose facial expression images is limited,which is difficult to meet the training requirements of large-scale deep neural networks.Thirdly,with the large-scale popularization of personal camera devices such as mobile phones,numerous multi-pose expression images are produced on the Internet every time.How to efficiently and conveniently use these image data will provide an effective technical way for facial expression recognition to solve the above problems.To solve the above problems,this paper tries to solve the problem of cross-angle facial expression recognition and generation from the perspective of domain adaptation.Through the research of generative adversarial training and semi-supervised learning,we can simultaneously generate and recognize cross-angle facial expressions in images in an unsupervised way.The contents of this paper can be summarized as follows:(1)In this paper,the current situation and existing problems of facial expression recognition are studied and analyzed.This paper introduces facial expression recognition methods and commonly used facial expression data,and points out the deficiency of current facial expression recognition algorithms.(2)The commonly used semi-supervised learning methods and generative adversity-network models are investigated,and the role and significance of semi-supervised learning in cross-angle facial recognition are briefly described.The classification experiments on two facial expression datasets are carried out to prove the improvement of the classification results of semi-supervised learning algorithm.At the same time,the principle and development of generative adversarial network are introduced,and the experiment of image style transformation is carried out based on the cycle generative adversarial network.(3)To reduce the domain differences between facial expressions from different angles,an generative adversarial network was used to extract high-dimensional features.The generative adversarial network in this paper is divided into two routes: one routes the source domain to the target domain,and the other is the opposite.While learning the mapping relationship between the two,the generative adversarial network also maintains the semantic information of expression before and after generation.In this process,a modified residual block is proposed to extract the high-dimensional features of three channels.(4)This paper proposes an unsupervised cross-view facial expression adaption network(UCFEAN).The main idea is to transform the unsupervised domain between two image spaces with different appearance adaptively into semi-supervised learning in the feature space with the same semantic content.UCFEAN projects the unlabeled target domain image and labeled source domain image into the feature space with the same semantic content through generative adversarial network,which meets the requirement of unsupervised feature learning that the features of the two domains are close enough.Then we can train the features of the target domain projection by using the label of the source domain projection in the feature space.(5)A large number of experiments were carried out on two multi-angle facial expression datasets,including qualitative evaluation,quantitative evaluation and ablation experiment.Compared with other mainstream existing methods,the effectiveness of the proposed algorithm was fully proved by the generated image quality and recognition accuracy.In this paper,UCFEAN is evaluated on two multi-view facial expression image databases,and it is obviously superior to other methods,which confirms the effectiveness of the proposed algorithm.At the end of this paper,the existing unsupervised cross-domain facial expression recognition work is further prospected.