Research On Robust Landmark Localization And Feature Extraction Algorithms For Micro-expression

Micro-expression is a kind of special facial minor movement.It is usually a short-lived emotion that people try to hide in some circumstances and cannot control by themselves.It can be used as an important basis for judging emotion.Relevant research is an important research direction in the field of emotional computing.And has a wide range of application prospects.The effects of short duration,low intensity of change,local asymmetry and environmental factors make detection and recognition of micro-expression very difficult.To this end,this paper proposes a robust landmark localization and feature extraction algorithms for micro-expression,and they are appliedto facial micro-expression recognition.The work of the paper is as follows:(1)Three facial landmark localization network models based on deep learning are studied and analyzed,namely coarse to fine convolutional neural network(Coarse to Fine CNN),multi-task convolutional neural network(Multi-Task CNN),style aggregation network(Style Aggregated Network,SAN).Comparative experiments in the 300W dataset and real environment show that compared with two other network models,the SAN model is more robust to changes in the environment and posture.Therefore,this paper selects the SAN-based facial landmark localization algorithm for the micro-expression feature extraction and recognition algorithm proposed in this paper.(2)A micro-expression feature extraction and recognition algorithm for sample distribution equalization constraints is proposed.Micro-expressions are difficult to capture and imitate,therefore,the existing related datasets all have the problem of unbalanced sample distribution and this situation will affect model training.In order to solve this problem,this paper designs a sample distribution equalization loss,and uses VGG-16-V as the skeleton network,and proposes micro-expression feature extraction and recognition algorithm based on sample distribution equalization constraints.Experiments in the CASME II dataset show that compared with the VGG-16-V network,the recognition rate of the proposed algorithm can be increased by 0.10%when the original sample is used as the input.When using the facial landmark to enhance the data level of the sample,the recognition rate can be increased by 10.10%.(3)A two-stream micro-expression feature extraction and recognition algorithm based on spatial-temporal jointly description is proposed.For a temporal relationship description stream,this paper designs a two-dimensional landmark feature extraction model(Two-dimensional landmark feature,TDLF)to extract the change of the distances between the landmark of the apex frame and the landmark of each frame(except the apex frame).And the spatial feature description stream selects the network structure of the sample distribution equalization constraint proposed in this paper.The temporal relationship and the spatial feature description stream predict output.Finally,the decision fusion is performed through the fully connected layer,and the proposed algorithm can be trained in an end-to-end manner.Experiments in the CASME II dataset show that the correct recognition rate of the proposed algorithm can reach 85.00%.