Multi-Pose Facial Expression Recognition In Natural Scenes

With the advent of the artificial intelligence era,facial expression recognition has become a very popular research topic,which can be applied to human-computer interaction,helping the elderly,helping the disabled,psychotherapy,intelligent monitoring and so on.However,most of the current facial expression recognition research is limited to the ideal state,and they can not be applied to the situation where the physical environment is complicated,especially the multi-pose and natural scene.Therefore,a high-descriptive and robust facial expression recognition algorithm is proposed for facial expression recognition in multi-pose and natural states,which is of great significance for human-computer interaction.In view of this,this dissertation has carried out an in-depth study on the combination of low-level and mid-level features,Convolutional Neural Networks(CNN)and Extreme Learning Machine(ELM).Then a feature extraction algorithm based on PHOG-LLC and a network structure based on CNN-ELM are proposed,and they are applied to facial expression recognition.The main work is as follows:Firstly,the state-of-art facial expression recognition technology are introduced from three aspects:facial expression recognition based on traditional methods,facial expression recognition based on deep learning and open facial expression database.And it also expounds the problems faced for facial expression recognition.Secondly,for the multi-pose problem,this dissertation proposes a facial expression recognition algorithm based on PHOG-LLC feature extraction.This algorithm combines the low-level and mid-level features,the global region and the local region.And a series of experiments were carried out in SDUMFE and Multi-PIE databases respectively,which verifies the high descriptiveness and robustness of this method.Thirdly,an expanded version of the Shandong University Multi-Pose Expression(SDUMFE+)database was created.There are seven facial expression categories:anger,happiness,sadness,surprise,disgust,fear,and twenty-one gestures in two degrees of freedom of pitch and yaw.Finally,aiming at the problems of multi-pose,illumination and occlusion in natural scenes,this dissertation proposes a facial expression recognition algorithm based on CNN-ELM network structure,which combines CNN and ELM classifiers effectively.And the performance of our method is compared with the existing mainstream deep network structure on the three databases:SDUMFE+,Multi-PIE and RAF-DB,which proves that our method is more effective.