Research On Micro-expression Recognition Based On Spatiotemporal Features Selection

The main difference between micro-expression and ordinary facial expression is that it cannot be controlled by human beings themselves,and it is the result of emotional suppression and repression,even human beings cannot perceive it.It can reflect human's real inner world and widely used in the medical diagnosis,investigation and interrogation,national security,and other fields.The amplitude of micro-expression is very subtle,and the existing micro-expression spatiotemporal feature extraction methods which extract the same micro-expression feature from the three different orthogonal planes and cannot accurately reflect the characteristics of different domains,so the recognition rates are low.According to the characteristics,that micro-expression sequence reflects motion and appearance information in temporal domain and spatial domain respectively,this thesis proposed a micro-expression recognition method by extracting different features from temporal domain and spatial domain.First,the micro-expression video is preprocessed by the Eulerian Video Magnification method and the face area is clipped out.Then the Improved Local Directional Number Pattern(ILDN)and the Pyramid of Histograms of Orientation Gradients Without Edge Extraction(PHOG-WEE)are extracted from the three orthogonal planes respectively.Final,the Support Vector Machine(SVM)is used to classify the micro-expressions.The mainly works in this thesis are as follows:This thesis proposes the ILDN model and uses it to characterize micro-expression in the temporal domain(XT,YT planes).The Local Direction Number Pattern(LDN)calculates the edge response direction through the edge strength and ignores the intensity information.The ILDN is proposed in this thesis,which fuses the comparison results among the pixel value of the main direction position and the value of the central pixel,then the direction information and the intensity information are extracted at the same time,which can enhance the discriminant power of the feature.PHOG-WEE is proposed in this thesis and used to characterize the video of micro-expression in the spatial domain(XY plane).According to the subtle characteristics of micro-expression,PHOG(Pyramid of Histograms of Orientation Gradients)is introduced into micro-expression recognition.And because subtle micro-expression can't use edge information to distinguish,the edge detection is removed when calculating the PHOG of the micro-expression image and only gradient features are calculated.A spatiotemporal micro-expression recognition method based on feature selection is proposed,which selects different feature extraction methods in temporal domain and spatial domain respectively.The ILDN in the temporal domain and the PHOG-WEE in the spatial domain are extracted respectively and then the feature histograms of three planes are concatenated to be as the final micro-expression feature.Compared with the state-of-the-art LBP-TOP,HOG-TOP,HIGO-TOP,LBP-SIP and Gabor algorithms on the SMIC,CASME and CASME II databases,experiments show that the proposed method has a better performance on describing the texture features of different planes and extracts accurate temporal and spatial features of micro-expression,which has a higher recognition rate.