Facial Micro-expression Recognition Based On Spatial-temporal Deep Learning

Micro-expressions are very short,uncontrollable facial expressions that humans reveal when they try to suppress or hide their true emotions.Therefore,it has been widely used in lie detection,police diagnosis,business negotiation,psychoanalysis and so on.The microexpression lasts only 1/25 to 1/2 second with relatively small face muscle movement,so it is very difficult to be recognized.To solve this problem,how to make full use of the characteristics of micro-expression and effectively recognize micro-expression has become one of the research hotspots in the field of facial emotion recognition.Previous studies have shown that,on the one hand,the traditional hand-crafted features can effectively extract the features of micro-expression,and achieve good performance in micro-expression recognition;on the other hand,convolutional neural network has shown strong image feature extraction ability in many fields of computer vision,such as image classification,object recognition,behavior analysis,etc.Based on these,in order to solve the problem of microexpression recognition,this paper proposes two micro-expression recognition algorithms based on convolution neural network.The specific work is as follows:(1)To solve the problem of poor expression ability of hand-crafted features,this paper constructs a CNN model with pyramid structure to extract the CNN features of the peak frame in a micro-expression video sequence.At the same time,LBP-TOP descriptor is used to extract the dynamic texture features of a video sequence.Finally,support vector machine is used to recognize the micro-expression.(2)In order to obtain the temporal correlation between the frames of the micro-expression video sequence,this paper proposes a micro-expression recognition algorithm based on the combination of CNN model of VGG-16 network and Long Short-Term Memory(LSTM)network,which is called CNN-LSTM deep network model.CNN in this network can extract the spatial domain features of a micro-expression video,while LSTM network can extract the temporal domain features of a micro-expression video,thus the temporal and spatial features of micro expressions are obtained.Finally,the output of each unit of the LSTM network is averaged,and the micro-expression is identified by a full connection layer.In the CASME II micro-expression open dataset,the above two algorithms are tested respectively.The experimental results show that the two algorithms proposed in this paper have better performance in micro-expression recognition.