Facial Expression Recognition Based On Deep Learning

As an important face attribute,expression has broad application prospects in the areas of human-computer interaction,communicational entertainment,intelligent robots and psychological crime.Expression recognition,as an important biometric recognition technology,has become a hot research topic in the feld of pattern recognition and computer vision.The definition of human expression recognition is to automatically recognize the true expression label based on the input face image by using computer vision and other technologies.Although many researchers have made arduous efforts to solve the human expression recognition,it is far from being solved and still faces many severe difficulties and challenges.Firstly,expression is a continuous dynamic change process.The same expression sequence can be divided into significant expression and weak expression according to the difference of expression intensity.Weak expressions under different categories have little difference,therefore it is very difficult to describe by artificially designed effective facial features.Secondly,the appearance of different people is different,and there are many similarities between the expressions of different categories.In order to minimize the disturbance caused by different people's appearance,it is crucial to accurately depict and extract the dynamic process in the same kind of expression.More and more attention has been paid to how to make full use of various effective temporal features in the dynamic change of the same kind of expression to identify the expression.Finally,it is also a very promising topic that how the expression analysis technology is applied and how to generate corresponding value in real life.To solve the above problems,a series of algorithms are proposed.The main work and contribution of this paper are summarized as follows:Deeper Cascaded Peak-piloted Network(DCPN)is proposed.In this paper,we proposes a DCPN algorithm based on weak facial expression recognition which has following advantages:(1)The technique of DCPN has three main aspects: Peak-piloted feature transformation,which utilizes the peak expression(easy samples)to supervise the non-peak expression(hard samples)of the same type and subject;(2)the back-propagation algorithm is specially designed such that the intermediate-layer feature maps of non-peak expression are close to those of the corresponding peak expression;and(3)an novel integration training method,cascaded fine-tune,is proposed to prevent the network from overfitting.Experimental results on two popular facial expression databases show the superiority of the proposed DCPN over state-of-the-art methods.Spatio-Temporal Convolutional features with Nested LSTM(STC-NLSTM)is proposed.A novel end-to-end architecture termed STC-NLSTM,which learns the muti-level appearance features and temporal dynamics of facial expressions in a joint fashion.Experimental results on four benchmark databases show that the proposed method achieves a superior performance to the state-of-the-art methods.A real-time facial expression analysis system is designed.Face detection and keypoint detection algorithms based on Muti-task Cascaded Convolutional Network(MTCNN)are integrated into the system.The real-time facial expression recognition module adopts the Inception-W algorithm proposed in chapter 2 as the internal integration algorithm,which can simultaneously conduct real-time facial expression intensity recognition and facial expression category recognition for the detected face images.