Research On Micro-expression Detection And Recognition Technology Based On Convolutional Neural Network

Micro-expression is a short-duration,small-movement and involuntary facial expression.It usually occurs when a person deliberately or unconsciously hides his or her true emotions.This provides a theory for revealing people's true psychology or emotions.in accordance with.Because micro-expression can reveal people's true psychological state,it has become an important subject in the understanding of human emotions and emotional phenomena,and has been explored by various disciplines such as psychology,sociology,neuroscience,and computer vision.These skills have practical application significance for psychotherapists,interviewers,and anyone engaged in communication.It has important applications in criminal investigation and trial,security protection,psychological consultation and other fieldsWith the rapid development of computer vision technology,micro-expression detection and recognition technologies are also emerging one after another.More and more technologies are applied to micro-expression detection and recognition,but most of the micro-expression detection and recognition algorithms still have shortcomings.The detection methods of the start and end frames mostly use the maximum duration of the micro-expression sequence as the size of the sliding window,or locate the climax frame,and then take a certain interval before and.after the climax frame as the start and end frames to fix the duration of the micro-expression;and Regarding micro-expression detection as a classification task,it cannot reflect the extent to which the candidate sliding window belongs to the micro-expression segment,resulting in inaccurate prediction of the start frame and the end frame.The biggest difficulty in micro-expression recognition lies in the limited number of samples in the database and the subtle differences between the samples,making the recognition rate unable to be further improved.In order to solve the above problems,this paper uses the BERT network and the three-dimensional convolutional neural network to learn the more subtle features of the micro-expression sequence,and uses the grouping algorithm to.more accurately locate the start and end frames of the micro-expression.With the help of the idea of comparative learning,learning the shared information of different modalities makes it possible to learn stronger feature expressions in the case of limited data samples.Specifically,the main contributions are as follows:?This paper proposes a micro-expression detection algorithm based on the BERT network.The algorithm mainly includes three parts:candidate segment generation module,spatio-temporal feature extraction module and grouping module.Specifically,the candidate segment generation module is used to divide the micro-expression sequence into several small-size candidate micro-expression segments;In order to learn the subtle features in the micro-expression segment,the spatio-temporal feature extraction module divides the candidate segments into different time slots,uses the three-dimensional convolutional neural network to learn the features in the time slot,and uses the BERT network to learn the features between the time slots;At the same time,this article is for accurately portray the degree to which the candidate segment belongs to the micro-expression segment,and adopt a regression loss optimization model;In order to ensure accurate positioning of the start frame and end frame position of the micro-expression,this paper uses a grouping algorithm to merge consecutive candidate segments and suppress overlapping segments.A large number of experiments have been carried out on CASME ? and SDU_spotting,and the results show that this method has higher detection accuracy.? This paper proposes a micro-expression recognition network based on bimodal contrast learning.The network mainly includes three modules:bimodal feature extraction module,bimodal contrast learning fusion modality and classification recognition module..Specifically,because the number of database samples is limited,and the samples not only contain the characteristics of the emotion category,but also the common characteristics between the samples.First,the micro-expression sequence is divided into RGB sequence and optical flow sequence;then the contrast loss between the RGB sequence and the optical flow sequence is constructed by contrast learning,so that the network can learn the general characteristics of the two modalities;in order to learn the category information of the micro-expression,this article will the bimodal features are fused,and the model is optimized using label data.It is ensured that while the network learns the shared information between the two modalities through comparative learning,it can also learn different types of features through supervised learning.This method has been extensively tested on CASME ?,SAMM and MMEW to verify the effectiveness of the proposed algorithm.