Research On Spontaneous Facial Micro-Expression Recognition

Facial Micro-Expressions(MEs)are spontaneous,involuntary facial movements when a person experiences an emotion but deliberately or unconsciously attempts to conceal his or her genuine emotions.Recently,ME recognition has attracted increasing attention due to its potential applications such as clinical diagnosis,business negotiation,interrogations,and security.However,it is expensive to build large scale ME datasets,mainly due to the difficulty of inducing spontaneous MEs.This limits the application of deep learning techniques which require lots of training data.In this paper,we propose a simple,efficient yet robust descriptor called Extended Local Binary Patterns on Three Orthogonal Planes(ELBPTOP)for ME recognition.ELBPTOP consists of three complementary binary descriptors: LBPTOP and two novel ones Radial Difference LBPTOP(RDLBPTOP)and Angular Difference LBPTOP(ADLBPTOP),which explore the local second order information along the radial and angular directions contained in ME video sequences.ELBPTOP is a novel ME descriptor inspired by the unique and subtle facial movements.It is computationally efficient and only marginally increases the cost of computing LBPTOP,yet is extremely effective for ME recognition.In addition,by firstly introducing Whitened Principal Component Analysis(WPCA)to ME recognition,we can further obtain more compact and discriminative feature representations,and then achieve significantly computational savings.In order to further improve the accuracy of micro-expression recognition,this paper proposes a method of Spatiotemporal Local Binary Pattern with Amplified Integral Projection and Multi-Cluster Feature Selection.The method first decomposes the image based on the Gaussian pyramid to different spatial frequencies,and then obtains motion changes through bandpass filtering.Using the integral projection method for amplified motion changes to obtain horizontal and vertical projections,the shape properties of the facial image can be preserved and the distinction between microexpressions can be increased.The local binary mode operator is used to extract the appearance and motion characteristics of the horizontal and vertical directions.Then,more discriminative features are selected by multi-cluster feature selection.Extensive experimental evaluation on three popular spontaneous ME datasets SMIC,CASME ? and SAMM show that our proposed approaches significantly outperform the previous state of the art on all three single evaluated datasets and achieves promising results on cross-database recognition.