Facial Expression Recognition Under Various Facial Posture

In the past decade,computer vision including facial expression recognition(FER)has gained outstanding achievements with the rapid developments of deep learning.As a result,FER is gradually applied in various fields such as education,medicine and security,etc Although the performance of FER has been effectively improved by deep learning based algorithms,it still faces many challenges for the realization of real reliable performance Considering the negative influences of complex scenes for FER in practical applications,this thesis studies the multi-view FER,and a expression recognition system is designed or FER under different face deflections.Firstly,a general lightweight FER algorithm is proposed in this thesis.By adapting the spatial affine transformation as image preprocessing,the deflected faces can be aligned such that the ability of convolutional neural network(CNN)for the facial key features extraction is improved.Meanwhile,considering the complexity of the deep learning based model,a fully convolutional networks(FCN)is designed for expression recognition.Combining the feature extraction modules of mulit-scale receptive field with the global average pooling layer,the proposed method can greatly reduce the model parameters and ensure the performance of FER.Then,to solve the problem of feature loss caused by face deflections,a face merged generative adversarial network(GAN)with tripartite adversaries is proposed for face frontalization in this thesis.With the introduction of an additional generator,the training intensity of the network is enhanced,which can effectively improve the synthesis performance.The first generator simultaneously focuses on the upper and lower parts of faces to capture the detailed features,and then the obtained high-dimensional decoded features together with the multi-scale facial encoded features are adapted as the input for the second generator.In the continuous confrontation between the two generators and the discriminator,the corresponding faces under different deflections are synthesized.In addition,the original faces are encoded once by the pretrained network,and the extracted features are shared by the decoders in the two generators,which can efficiently reduce the complexity of the model and the difficulty involved in the training procedure.Finally,combining facial expression classifier and face frontalization algorithm,a GAN based three-stage-training algorithm is proposed for multi-view FER.In the first stage,the samples of frontal faces are trained for the expression classifier,which is added to expression keeping GAN in the second stage to ensure the expression consistency between the synthesized and original faces.In order to make full use of the original facial expression information,the original faces and their corresponding synthetic ones are fused to retrain the classifier in the third stage,so as to implement the multi-view FER.For the CNN based model,the three-stage-training can improve the performance effectively,which solves the problem of the poor feature extraction for the facial deflections.Comparing with some existing methods,experimental results show that the three-stage-training algorithm can achieve the best recognition accuracy.