A Micro-expression Recognition Algorithm Based On 3D Convolutional Neural Network And Optical Flow Fields From The Neighboring Frames Of The Apex Frame

Different from macro facial expressions,micro expressions are subtle and unconscious facial expressions.Macro facial expressions are sometimes deceptive,but micro-expressions reflect a person's true emotions.Because of its short duration and low intensity,micro-expressions are difficult to detect and explain correctly,so the accurate recognition of micro-expressions is still challenging.Micro-expression recognition is an emerging research field,which is currently in the early stages of development.We propose a 3D-Apexflow micro-expression recognition algorithm that focuses on the optical flow information near the micro-expression apex frame.First,we obtain the position of the apex frame of each micro-expression video sequence which depicts the highest intensity of facial motion in all frames.Then,the optical flow fields of the apex frame and the left and right 4 frames of images are extracted and fed to a 3D convolutional network for further feature enhancement,finally micro-expression classification is performed.Comparison experiments show significant improvement of the accuracy rate over original image frame input,and the recognition time is significantly reduced.Experimental results also show that reducing the redundant information of non-apex frames by focusing on the apex frame improves robustness.The proposed network model can obtain higher-level features from optical flow sequence.In order to analyze the performance gains brought by the algorithm of this paper in order to locate the apex frame and extract the optical flow separately,we designed a 3D-Apex model and a 3D-Flow model to do ablation experiments for verification.For evaluation of the effectiveness of 3D-Apexflow,we conducted extensive experiments on three public spontaneous micro-expression data sets:CASMEII,SMIC,and SAMM.Compared with other comparative experiments,the proposed 3D-Apexflow micro-expression recognition algorithm achieves a better recognition performance.