A Study Of Facial Expression Recognition Based On Representation Learning

Facial expressions are an important way for human beings to express and convey emotions.Automatic facial expression recognition is essential for machines to perceive emotions of users.Existing facial expression recognition methods often use handcrafted representations or data-driven representations learned by deep learning technology,and fail to effectively integrate facial prior knowledge.In addition,the existing robust facial expression recognition research fails to make full use of the relationship between the user and the facial expression contained in the image sequence to disentangle user information from facial expression information.This thesis focuses on the representation learning for facial expression recognition and performs the study as follows:1.This thesis proposes a attention-based facial expression recognition method enhanced by the prior knowledge.Firstly,the domain knowledge of expression is systematically summarized,namely the symbiotic relationship between facial expression and facial region as well as that among faical regions.The order relationship of attention between the global and the local,as well as that among the local regions,is defined by the aforementioned symbiotic relationship,which can restrict the process of attention learning.Secondly,a representation extraction network is constructed to learn global and local representations of the face,and then the self-attention learning process of the local representations is constrained according to the order relationship of attention defined among the local regions.Nextly,the reconstructive local representations fused with this constraint are obtained.Then,according to the order relationship of attention between the global and the local,the attention learning process between the global representation and the new local representation is constrained,so as to obtain the expression representation that incorporates domain knowledge.Finally,the global representation is fused with the facial expression representation constrained by domain knowledge for facial expression recognition.The experimental results on the two data sets,i.e,CK+and Oulu-CASIA,verify the effectiveness of the method.2.This thesis proposes a facial expression recognition method based on representation decoupling and image sequences.First of all,based on the information about expression intensity and user in the image sequence,two representation extraction networks are constructed,where the expression representations of adjacent frames from the same expression are close in the expression representation space and the attribute representations of video frames with same users are close in the attribute representation space.Secondly,reconstruction learning and adversarial learning are introduced to generate the corresponding and realistic facial expression images by combining the facial expression representations and user attribute representations from different users,so as to decouple the facial expression representations from the user attribute representations.Finally,the user-robust expression representations are decoupled for expression recognition.Experimental results on three data sets,i.e,CK,Oulu-CASIA and MMI,verify the effectiveness of the proposed method.