Research On Micro-expression Recognition Technology Based On Spatio-temporal Features

Micro-expressions are an essential part of human non-verbal communication.In daily life,for various reasons,people unconsciously express their true internal thoughts by performing tiny facial changes,and a sensitive observer is frequently able to make corresponding decisions with relevant information.Therefore,by analyzing the psychological activities underlying human behavior,we can better understand the relevant social phenomena,which has significant application in fields such as medicine and criminal investigation.To explore the nature of micro-expressions in depth,the main research of this paper is as follows:(1)To boost the motion information in micro-expression images,a deep learning-based motion amplification technique can be used to amplify the difficult-to-perceive and capture motion details in them to an appropriate scale,which provides more valuable input data for subsequent tasks.Since the number of samples in the current publicly accessible microexpression dataset is small,expanding the training samples using data augmentation is effective in both reducing the risk of overfitting and enhancing the generalization ability and stability of the model.(2)To address the problems of existing deep learning-based micro-expression recognition models,which have too many network parameters,we propose a micro-expression recognition model that utilizes a lightweight network as the backbone network and incorporates an attention module.The use of lightweight network can realize faster inference speed and lower computational cost at the expense of a minor amount of accuracy.Meanwhile,the introduction of the CBAM module can make the more expressive micro-expression features valued,and replace the large convolution kernel in its spatial attention module with a dilated convolution,which could both better preserve the spatial features of micro-expressions and effectively reduce the network parameters.The improved CBAM module is noted as the DCBAM module and is fused into the main structure of the lightweight network for better integration of contextual information.The experimental results demonstrate that the best combination of micro-expression recognition algorithm based on lightweight network is the combined Efficient Net-B3+DCBAM scheme,which improves the recognition accuracy by 1.43% with basically the same amount of computation.(3)To capture the spatial and temporal changes in micro-expressions more precisely,a micro-expression recognition algorithm based on spatial and temporal features is proposed,which constructs a temporal feature extraction network and a spatial feature extraction network,respectively.Among them,the temporal feature extraction network is implemented using a recurrent gating unit GRU,replacing the dot product operation in the GRU with a convolution operation and selecting the optimal activation function to enhance the nonlinear expression of micro-expression features,and the improved GRU is denoted as HConv GRU.the spatial feature extraction network is implemented using the recognition network based on lightweight networks as described above.The above two-branch network is used to extract the temporal features and spatial features of micro-expressions separately,and the features are fused for both,and the fused spatio-temporal features are used to achieve the recognition of micro-expressions.The experimental results show that comparing with the single spatial or temporal features of micro-expressions,the recognition accuracy of micro-expression recognition using the fused spatio-temporal features is improved by 2.26% and 6.21%,respectively,and the effectiveness of the micro-expression recognition algorithm based on spatio-temporal features is verified.