Facial Expression Recognition Based On Deep Learning

In recent years,facial expression recognition(FER)has become a popular topic with the promotion of biometrics technologies.FER is considered in a wide range of applications,such as human-computer interaction,safe driving,assisted medical care,and online education.There are two key steps in the traditional,feature extraction and feature learning algorithm.Recently,deep learning algorithm have shown significant performance improvements for facial expression recognition due to its strong representation power.Most deep learning based methods,however,focus more attention on spatial appearance features for classification,discarding much useful temporal information.As we known,the behavior of expressions is a dynamic process,and the temporal dynamics of facial expressions are crucial for FER.Making use of sequence images of expressions for research can greatly improve the accuracy of recognition and has a great research value.In the general case,the method of FER can be roughly divided into two categories,imagebased method and sequence-based method.In this work,we proposed a novel deep network which learns spatial features for image-based FER.It is based on a inception structure,which better suits the task at hand and hence leads to improved performance.In addition,as the dataset of FER is limited,we fine-tuned the framework several times to prevent the deep network from over-fitting.For the sequence-based expression recognition,we propose a novel framework which jointly learns spatial features and temporal dynamics for facial expression recognition.Given the image sequence of an expression,spatial features are extracted from each frame using a deep network.Whilst the temporal dynamics are modeled by a convolutional network,which takes a pair of consecutive frames as input.Finally,the framework accumulates clues from fused features by a BiLSTM network.In addition,the framework is end-to-end learnable and thus temporal information can be adapted to complement spatial features.Experimental results on two benchmark databases,CK+ and Oulu-CASIA,show that the proposed deep network improves the accuracy of image-based expression recognition.For sequence-based expression recognition,experimental results on three benchmark databases,CK+,Oulu-CASIA and MMI,show that the proposed framework outperforms state-of-the-art methods.