Research On Facial Micro-expression Recognition Technology Based On Deep Learning

With the increasing needs of all aspects of society and the development of information technology and artificial intelligence,the field of micro-expression recognition has attracted more and more researchers’ attention.As a spontaneous and non-deceptive facial expression,micro-expression can provide a strong basis for judgment in the fields of criminal trial,psychotherapy,public safety and so on.Short duration and small motion range are not only the remarkable characteristics of micro-expression,but also the difficulties of microexpression recognition.In addition,because it is difficult to capture spontaneous microexpressions in the process of data collection and face data involves personal privacy,the number of samples in the existing public datasets is relatively small.To solve the above problems,this thesis mainly studies the methods based on video motion amplification technology,convolution neural network and different motion feature fusion to improve the accuracy of micro-expression recognition based on deep learning model.Specifically,the main research contents are as follows:(1)Aiming at the problems of small motion range and small dataset samples of micro-expression,the convolution neural network based on transfer learning is proposed for micro-expression recognition.The macro-expression recognition network model is improved by adding projection transformation and channel attention mechanism,and the new network model is constructed to adapt to the micro-expression recognition task.Projection transformation reorganizes the input features,which can not only produce more distinguishing features,but also reduce the network parameters.The channel attention mechanism helps the network further pay attention to and select the information that is helpful to the classification of micro-expressions.The proposed network model has less training parameters and less computation,which can effectively avoid over fitting.In order to enhance the similarity between micro-expression and macro-expression during learning,the motion of microexpression is adaptively enlarged and the intensity of macro-expression is reduced.Finally,validation experiments are carried out on multiple micro-expression datasets,and the results show the effectiveness of the proposed method.(2)Aiming at the problem that the recognition accuracy of a single optical flow feature greatly affected by age and illumination changes is reduced due to noise,the two-stream convolution neural network based on multi motion feature fusion is proposed for microexpression recognition.The facial landmark feature map which are not easy to be affected by the change of age and illumination and optical flow feature are fused,the two-stream convolution neural network with multi motion feature fusion is constructed,and the classification fusion decision is added at the end of the network to improve the recognition accuracy of the network model.Aiming at the problem that the action unit information given by the label of micro-expression dataset is not used to assist micro-expression recognition,the landmark feature map is weighted according to the close relationship between the action unit and the landmarks.Finally,experiments on a series of micro-expression datasets show that the proposed method can improve the recognition accuracy.(3)The novel coronavirus pneumonia and academic research pressure have a dual impact on postgraduate students,and the number of graduate students suffering from depression and suicidal tendency increased significantly.In order to implement the research content of this thesis in the practical application scenario,the Jiangnan University’s micro-expression dataset is established,and the emotion state analysis system for graduate students is built.The system has the functions of single sample micro-expression recognition,micro-expression data management,subject information management and personal emotional state analysis.The micro-expression recognition algorithm proposed is applied to the single sample microexpression recognition and personal emotional state analysis module.