Multi-dimensional Facial Expression Recognition For Emotional Cognition

Since deep learning,big data,cloud computing and other artificial intelligence technologies have become one of the most popular research topics.Facial expression recognition,as one of the important branches of AI,enables the computer machine to recognize the user's emotional state through intelligent computing,which has great significance in enhancing the intelligence and friendliness of human-computer interaction.This thesis focuses on the analysis of facial expression recognition through multidimension,which refers to the plane dimension and temporal dimension.The main tasks of this thesis are as follows:(1)The research object of plane dimension is the static single facial expression image,the content of which is usually the moment of the most abundant expression of a certain expression.The deep learning is the most effective way to deal with facial expression recognition.However,the fatal problem of deep network is that the training must have large-scale data,otherwise it wound be easily became overfitting.Through careful observation,we find that the same area of different faces has similar emotional characteristics under the same expression.Therefore,in order to expanding the training dataset,this thesis designs a facial local area swap generative adversarial network.This network aim at replacing the interesting areas of different faces and fusing into new facial images,so as to eliminating facial identity features and expanding the scale of training data set.The greatest advantage of the network is that it can processing the wild face data set so that it can improve the robustness of deep learning models.(2)The research object of temporal dimension is the dynamic facial image sequences,which continuously have one single expression catgory.Dynamic facial sequences expression recognition are more complex that static single expression recognition in data dimension and feature dimension.Because it not only needs to extract images spatial domain features,but also needs to consider temporal features.In the thesis,we designs a combination of shallow feature and deep feature with attention mechanism.In order to extract the shallow feature,a shallow attention model is proposed to describe the action unit of FACS by using the relative position of facial landmarks and the texture features of local facial areas.At the same time,the advantage of deep convolutional neural network in expressing high-level semantic features is used to extract deep features of sequential face images.Finally,the shallow model and the deep model are integrated into the multi-attention shallow model and the deep model to complete the dynamic sequence facial expression recognition.We validated our dynamic expression recognition system on three publicly available databases,CK +,MMI and Oulu-CASIA,and achieved better performance than other recent results.