Facial Expression Feature Learning Method Based On Deep Convolutional Network

Emotion is human's subjective feeling about objective existence,and facial expression is an effective way to express emotions.The interaction between human and machine will be better improved if facial expression can be recognized accurately.Therefore,the task has a wide application,and attracts a lot of researchers to do a series of experiments on it.We propose two feature learning methods for static facial images and dynamic sequences to improve recognition accuracy.A method for static facial expression recognition based on deep facial landmark feature is proposed.Based on the relationship between facial expression recognition and facial landmark localization,we propose to improve the recognition accuracy by adding the information of facial landmarks which indicate the shape of facial organs.Specifically,we propose a multitask convolutional neural network,recognizing facial expressions and locating facial landmarks simultaneously,which makes the model explore the intrinsic correlation between the two tasks during optimization.Furthermore,a location attention map is generated by means of detected landmarks to adjust the weights of features extracted from different facial regions.In this case,features around facial organs will be emphasized while features extracted from edge regions will be suppressed.Experiments on three popular datasets(CK+,Oulu-CASIA and MMI)prove the effectiveness of proposed method.A method for dynamic facial expression recognition(DFER)based on phase space reconstruction(PSR)is proposed.Through observation,we find that spatial texture and temporal dynamics of dynamic facial expression are highly correlated,and they should not be extracted separately.Moreover,it is essential for DFER to capture the gradual process of expressions from individual frames.Based on these analysis,we utilize three-dimension convolutional neural network(C3D)to extract spatial and short-term features simultaneously,which are termed as observations in the phase space.A new phase space reconstruction(PSR)method is proposed to reconstruct spatio-temporal features extracted by C3 D so that reconstructed observations can effectively represent the gradual process of dynamic facial expressions.Reconstructed observations contain abundant temporal information,which is beneficial to DFER.Experiments on three popular datasets(CK+,Oulu-CASIA and MMI)prove the effectiveness of proposed method,and visualization of heatmaps demonstrates that reconstructed features have global consistency in facial regions and find the underlying evolutionary pattern of dynamic facial expression.