Research On Facial Expression Recognition Technology Based On Deep Metric Learning

Facial expression is the most powerful and natural means of understanding human emotional expression.Automated facial expression recognition(FER)attempts to classify faces in a given single image into one of seven basic emotions.Recently,deep learning-based FER methods have been successful in classifying posed facial expressions in a controlled environment.However,these methods can not properly classify images captured in the wild.This is mainly due to facial expression in real world is usually unbalanced distributed,and it is susceptible to interference from head posture,illumination,occlusion,and personal attributes.FER in the wild is closer to real-world applications.Therefore,it is of great significance to solve the problem of facial expression recognition in the wild.In summary,the main contributions of this thesis are as follow:(1)A novel occlusion expression inpainting model named EIGAN is proposed to repair the partial occlusion of facial expression images in real world.EIGAN,based on the generative adversarial networks,is composed of a generator,a local discriminator and a global discriminator,and expression labels are added as constraints.The local discriminator is used to complete the masked region of the image,and the global discriminator is used to ensure the global consistency of the entire image.Experiments show that the completed image generated by EIGAN not only ensures the authenticity of the filling part and the consistency of the entire image,but also retains the original semantics of the image(2)A novel quadruplet-mean loss is proposed to solve the inra-and inter-variations of facial expression images in real world.The quadruplet-mean loss learns a center for the deep features of each category,reduces the distance between the positive sample and the positive center,increases the distance between the positive sample and the negative center,and requires the distance between different centers greater than the distance between samples of the same category.It can keep the intra-class compactness as well as inter-class dispersion,thereby enhancing the ability to distinguish features.(3)A novel balanced-softmax loss is proposed to eliminate the limited recognition performance caused by imbalance between different classes in FER.The balanced-softmax loss,improved by softmax loss,sets different loss weights for different classes according to their misclassification rates.By punishing heavily on difficult samples with a small amount of data,difficult samples can affect network parameters more in the network training process,which can enhance the recognition of difficult samples and achieve better performance.(4)A novel FER method termed DFER-Net is developed to enhance the discriminative power of facial expression recognition.By combining quadruplet-mean loss and balancedsoftmax loss,DFER-Net pays more attention to the minority class and continuously optimize the intra-class distance of similar expressions and the inter-class distance of different expressions.As a result,the network can extract features with stronger recognition.Through experiments on FER2013 and RAF-DB datasets,the algorithm of our thesis is comprehensively evaluated.The experimental results indicate that both the EIGAN model and the DFER-Net model have good performance on these two datasets.The combined use of EIGAN model and DFER-Net model proves the effectiveness of recognizing the occluded image after repairing.