Deep Learning-Based Micro-Expression Magnification And Recognition

Existing research shows that detecting subtle intensity changes and finding the internal relationships of local features in facial images are two key issues for microexpression recognition.In order to address the problem of subtle intensity changes in micro-expressions,existing research has introduced motion magnification technology.However,this technique is mostly a traditional method that requires professional knowledge and complex parameter adjustment.Therefore,this thesis proposes a microexpression magnification network based on deep learning to simplify the process.Regarding the extraction of internal relationships between local features of microexpressions,a self-attention mechanism is introduced into micro-expression recognition to extract geometric relations between local pixels.The work of this thesis is as follows:(1)To cope with the subtle intensity changes of micro-expressions,this thesis designs the Cascade Micro-Expression Magnification Network(Cascade-MEMN)to amplify the intensity of micro-expressions.It uses the macro-expression dataset to train the model and applies the training results to microexpression magnification to solve the problem of the magnification network training without labels.Additionally,the thesis introduces the progressive cascade method into the magnification network to solve the problem of image distortion and artifacts caused by a large magnification factor.The progressive cascade method consists of three networks with the same structure but not sharing weights,and it decomposes large magnification factors into several small ones to ensure the generation of high-quality magnified images.Furthermore,to address the issue of the cascade network’s difficulty in converging at the initial stage of training,this thesis introduces the Cycle-GAN network structure into the cascaded subnetwork.The Cycle-GAN network structure improves the utilization of the dataset and ensures the generation of reliable pre-training models for the subnetwork.(2)To find the local geometric relationship features of microexpressions,this thesis introduces the self-attention mechanism into the recognition network and proposes a Self-Attention-Based Convolution Layer(SCL).This layer introduces self-attention into each pixel in the window in the form of a sliding window,taking into account the geometric relationship between the local pixels of microexpressions.It can obtain more differentiated micro-expression features and improve the recognition rate of micro-expressions.(3)This thesis proposes a new micro-expression magnification and recognition model by combining the micro-expression magnification network with the self-attentional convolutional recognition network.To verify the reliability and feasibility of the model,this thesis applies the proposed micro-expression magnification recognition system to the composite database of MEGC 2019 for evaluation experiments.The results obtained are competitive with the existing algorithms,demonstrating that the proposed network model can improve the recognition rate of micro-expressions.