Research Of Deep Learning For Facial Expression Recognition

Facial expression can convey a wealth of emotional information.With the popularization of computer technology in people's daily life, facial expression recognition has more wider application prospect in many fileds, such as HCI, family entertainment, public safety and medical treatment. Recent years, the rapid development of Deep Learning has bring new opportunities to every research fileds. Unlike the traditional manual methods of extracting features, researcher will obtain the feature which is automatic learning and has strong generalization ability through Machine Learning methods.Taking into account the specificity of facial expression, we solve the facial expression recognition by Deep Learning.We adopt deep convolutional neural network of deep learning. In order to solve the dicculty in facial expression recognition, such as subtle changes in expression and confusion in different expression, we propose five networks.(1)In order to increase information in images by transforing gray to pseudo-color, we propose Peak Convolutional Neural Network, PCNN, which is a network based on static image and appearance feature.(2)In order to obtain dynamic information from changes of facial expression by image sequences, we propse 3D Appearance Neural Network, 3DANN, which is a network based on image sequence and appearance feature.(3) In order to obtain constaints of facial features by facial landmarks, we propose 3D Geometry Neural Network, 3DGNN, which is a network based on image sequence and geometry feature.(4) In order to combine the advantage of 3DANN and 3DGNN by combining two networks, we propose 3D Appearance-Geometry Neural Network, 3DAGN, which is a network based on image sequence, appearance and geometry feature.(5) In order to add the information of peak frame by adding PCNN, we propose Deep Peak-Appearance-Geometry Neural Network, DPAGN, which is combination of not only appearance and geometry but also static image and image sequence.According to the experiment results given in this paper, pseudo-color enhancement and 3D convolutional based on image sequence is benefit to facial expression recognition. Moreover, the proposed five networks can achieve high classification accuracy.