Research On Facial Expression Recognition Based On Deep Attention Mechanism

Facial expression,the most natural and direct way for humans to convey emotions,has always played an important role in the research of human emotion analysis.The ac-curate recognition of facial expressions is particularly important for the correct under-standing of human emotions.With the continuous advancement of software and hardware technologies and the development of artificial intelligence theory,facial expression recog-nition will be more practical and universality in many fields,such as assisted living,social entertainment,and human-computer interaction.Nowadays,there are lots of mature ap-plications of Facial Expression Recognition(FER)technology for the laboratory scene.However,the detection and recognition of facial expression in image,especially in the natural scene image,is an important but challenging task.It is still at the stage of research and exploration.The main work of this thesis are as follows:1.In this thesis,a expression images generation algorithm based on content and style decoupling is studied.Firstly,the action units information of various expressions is ex-tracted by FACS.To obtain the expression content and style codes,two different encoders are designed to decouple the content and style of the input expression images separately.Then the previously obtained action units information and style codes are fused.Com-bined with the adaptive instance normalization,the fusion information is used as the input to the decoder network,to control the generation of different expressions.The expression data obtained by this algorithm has enriched the expression recognition dataset and laid a solid foundation for further facial expression recognition.2.This thesis studies an attention mask generation algorithm based on clustering and gradient fusion.Leveraging on the multiple clustering algorithms,the clustering analysis of facial landmarks is conducted to obtain the general clustering characteristics of these landmarks.Then the face is divide into different regions based on the result of cluster-ing,the gradient histogram of each regions and the entire face image are calculated to build a probability distribution model,which is used to calculate the correlation coeffi-cient between each divided part and the overall image,to obtain the weight matrix of the non-key parts correspondingly.Finally,the weight matrix at each landmark point and its neighborhood is calculated and merged into the attention weight matrix to complete the construction of the attention mask?The expression image and the attention mask are mul-tiplied to obtain the corresponding mask image.3.An attention mechanism based FER algorithm is studied in this thesis.The pseudo-siamsis network is designed with two branches,which are the content branch and attention branch.The content branch is designed as a MobileNet-like structure to obtain semantic information.The attention branch uses the structure of a codec to obtain the structured information of the expression image.In terms of the multi-scale input image,this thesis utilizes the original expression image as the input of the content branch,and the mask im-age is used as the input of the attention branch.The outputs of these two branches are then fused on multiple scales to enhance the attention area.Meanwhile,the full convolution is applied to alleviate the problem of overfitting in fully connected networks.