Research On Robust Feature Extraction And Deep Modeling In Micro-Expression Recognition

With the development of the information-based society and image recognition technology,facial micro-expression recognition plays a crucial role in a wide range of applications of psychotherapy,security systems,marketing,commerce and much more.In contrast to facial macro-expressions,which normally lasts between 0.75 s to 2s,micro-expression usually occurs in less duration(0.04 s to 0.2s)and lower intensity.Exploration of automatic facial micro-expression recognition systems is relatively new in the computer vision domain.This is due to the difficulty in implementing optimal feature extraction methods to cope with the subtlety and brief motion characteristics of the expression.For micro-expression recognition,there has been a growing interest in incorporating computer vision techniques in automated micro-expression recognition systems.Fast,distinctive feature extraction and modeling method is one of the key issues for robust micro-expression recognition.Recently,deep convolutional neural networks(CNNs)have shown the great power in various fields and outperformed the traditional handcrafted features as well as shallow classifiers.It is an interesting tendency that CNN is applied to micro-expression recognition problem as features and classifiers.To get robust feature representation,this paper focuses on the temporal-spatial features extraction and deep modeling for micro-expression recognition.Our main contributions are summarized as follows:1.A micro-expression analysis framework with adaptive key-frame representation is founded to improve the effectiveness of the extraction of time-order characteristics.The structural similarity index is used to select the key-frame of the micro-expression sequence adaptively.The robust principal component analysis is used to obtain the information of the sparse key-frame.Also,dual-cross patterns are constructed for feature extraction.Experiments are conducted on the CASME2 and SMIC microexpression datasets,which have achieved recognition rates of 64.63% and 62.8% accordingly,and the recognition speed is also improved.2.To further improve the robustness of the extraction of spatial features,a new micro-expression method based on the local key-frame is proposed.The human face is divided into five regions of interest,and key frames are extracted from each region to obtain dual-cross patterns features.Finally,the features of each region are selectively connected to form the final feature vector.Experiments were performed on the CASME2 micro-expression data set,which achieved the highest recognition rate of 68.71%.This indicates that based on the method of local regions of interest,the position and structural information of each region of the face can be fully utilized,and the influence of irrelevant regions on micro-expression recognition can be effectively removed.3.In order to make full use of the deep features of micro-expressions,a deep model based on transfer learning is trained and set up.The structural similarity index is used to find the key-frame in the micro-expression sequence.And the differential image of the micro-expression sequence is obtained through the key-frame and the initial frame.Resnet-18 is pre-trained on the CK+,CASIA,fer2013 micro-expression datasets,and then the network parameters are transferred to the micro-expression to make fine adjustments.The experiments performed on the CASME2 and SMIC-HS micro-expression data set have achieved the recognition rates of 74.39% and 76.22%,which illustrates the applicability of transfer learning in micro-expression recognition based on a small data set.