Study On The Deep Feature Enhancement For Micro-expression Recognition

The development of computer vision technology and industry has put forward higher requirements for human-computer interaction functions.Recognizing microexpressions as a way of human-computer interaction,which can reflect the real intention of human inner,has great potential for applications in psychological research,defense security,business negotiation and education industries.The weak intensity and short duration of micro-expressions lead to their discriminative features that are difficult to be captured by algorithmic models.Therefore,this thesis studies the theory and methods related to micro-expression recognition from the perspectives of network attention mechanism and representative feature fusion based on deep feature enhancement techniques.The main work of this thesis includes the following aspects:(1)Multi-scale feature enhancement for micro-expression recognition.To address the problem that the intensity of micro-expression motion is weak and it is difficult for network models to distinguish it from noise,a lightweight attention-based multi-scale feature enhancement Network,LAMNet,is proposed.First,a lightweight attention model,LAM,is constructed for enhancing the network’s ability to characterize microexpression features.In this model,a one-dimensional convolution is used to map adjacent feature maps and calculate the correlation of channel features,and a doublelayer dilated convolution is used to update individual feature maps and calculate the correlation of spatial features.LAM is then combined with residual blocks to compute and update micro-expression feature map weight at different scales in LAMNet.In addition,to alleviate the network training overfitting,this thesis transfers the facial expression recognition knowledge to the micro-expression dataset for learning.Experiments are conducted using different evaluation methods to verify the sophistication and effectiveness of the network and modules.Two strategies visualize the modeling behavior of the network on micro-expression features.(2)Representative complementary features fusion for micro-expression recognition.In view of the different contributions of local regions of faces to micro-expressions and the complementary characteristics of image features and structural features,a feature enhancement network based on fusion of representative regions and structural relationships is proposed.First,the face is segmented based on the Delaunay Triangulation and Voronoi Diagram to obtain local regions that are consistent with the anatomy of the face.Secondly,the representative regions are filtered based on Pearson correlation coefficient statistics and the image features are extracted by the sub-network Dual-stream LAMNet.Then,to exclude face difference interference and capture the structural information,the graph structure representation of facial landmarks is modeled and the micro-expression feature is captured by the sub-network GCNNet.Finally,the FusionNet is constructed to fuse image features with structural features.In addition,this thesis separates representative regions according to muscle group distribution and employs dual-channel features learning to improve model flexibility.Experiments are conducted using different evaluation methods to verify the sophistication and effectiveness of the network and models.(3)Micro-expression-based class concentration analysis system.Combining existing digital image processing and object detection technologies,based on the theoretical research and related work of deep feature enhancement for micro-expression recognition in this thesis,a micro-expression recognition model and a fatigue action detection and analysis model are constructed respectively.By modeling the relationship between micro-expressions and fatigue movements,this thesis develops a microexpression-based class concentration analysis system,which can capture and identify micro-expression and analyze concentration based on micro-expressions and fatigue movements.