Research On Micro-expression Recognition Methods Based On Transfer Learning From Macro-expressions

Micro-expressions are spontaneous facial expressions when people are under effective psychological stimulations,which can reflect the most real emotion changes.Generally,micro-expressions appear with an extremely short duration and low intensity,resulting from the uncontrollable occurrence of them.Even if people try to hide their true emotions,small movements of facial muscles are still inevitable.Therefore,the most real innermost emotions of people can be revealed by micro-expressions and the applications of micro-expressions are irreplaceable in many aspects.For instance,the micro-expressions recognition is able to tell whether a person is lying when he/she is speaking.In other word,great application values of micro-expressions recognition can be found in various fields,such as judicial interrogation,criminal investigation,security,prison supervision,and psychological research.Moreover,micro-expressions recognition also starts to be applied in the fields of finance,education,entertainment,and etc.The successive emergences of interview credit system and smart classroom based on micro-expressions have indicated the great significance and development prospect of micro-expressions recognition.The technology of machine learning and pattern recognition has provided basic support for the development of micro-expressions recognition.So far,numerous researches work on the algorithms of micro-expressions recognition have emerged but most of them are unsatisfactory.On the one hand,the short duration and low intensity make the micro-expressions difficult to capture effectively.On the other hand,the number of available high-quality micro-expressions databases is very small at present.Moreover,the labeled samples in the databases are limited and uneven,which substantially stands in the way of building better micro-expressions recognition models.In order to address this issue,transfer learning is adapted to acquire auxiliary data for information compensation and data expansion.As we know,the samples of macro-expressions are rich and balanced.Meanwhile,like micro-expressions,macro-expressions can also reflect emotions thorough facial expressions,showing certain similarity between them.In conclusion,two novel micro-expressions recognition algorithms employing the macro-micro transfer are proposed in this study to solve the problem of insufficient micro-expressions labeled samples.Specifically,the main contributions of this paper are as follows:? A micro-expressions recognition model based on joint representation learning is proposed.Macro-expressions that share some common characteristics with micro-expressions are used as auxiliary domains to learn the common subspace,which minimizes the difference between macro-expressions and micro-expressions.Based on the micro-expression data retaining the manifold structure,the macro-expressions most similar to the micro-expressions are used for reconstruction,which helps improve the connection between macro-expressions and micro-expressions.In order to improve the robustness of the model,the residual term is added to the original model,which can be minimized to reduce effect of noise.Eventually,this proposed model is validated on a total of three cross-domain datasets(CK+macro-expression database&SMIC micro-expression database,CK+macro-expression database&CASMEII micro-expression database,and MMEW macro-expression databaseµ-expression database)using two representative features.? A joint non-negative matrix factorization micro-expressions recognition model based on dual graph regularization is proposed.The informative macro-expressions knowledge is also introduced to find common emotional factors from macro-expressions and micro-expressions themselves.First,the macro-expressions and micro-expressions are aligned and jointed,and the joint data of the two domains are processed with joint non-negative matrix factorization.At the same time,the dual graph relationship within and between the two domains is maintained,making the macro and micro-expressions more closely related.Eventually,this proposed model is validated on a total of three cross-domain datasets(CK+macro-expression database&SMIC micro-expression database,CK+macro-expression database&CASMEII micro-expression database,and MMEW macro-expression databaseµ-expression database)using two representative features.