Research On Micro-expression Recognition Technology Based On Feature Fusion

Micro-expression is a kind of spontaneous and uncontrolled special facial muscle movement,which can "reveal" a person's true feelings.It has a wide application value in the fields of national security and psychological consultation.Unlike ordinary macro expressions,which can be directly recognized by human eyes,even trained professionals can not achieve satisfactory results when recognizing micro-expressions.In order to improve the recognition rate of micro-expression and save labor cost,automatic recognition of micro-expression has gradually become the main research direction.At present,most researches on micro-expression only extract their single feature,which leads to incomplete information extracted from micro-expression,affecting the final recognition rate.In view of the above problems,this paper integrates different micro-expression features for recognition,and mainly does the following works:(1)Proposing a micro-expression recognition method based on feature selection and kernel canonical correlation analysis.The extracted LBP-TOP and HOG-TOP features are eliminated by feature recursive elimination method to remove redundant information,and then the two feature subsets are fused by kernel canonical correlation analysis method to obtain new KCCA-HL micro-expression features.Experiments are performing on the SMIC micro-expression database by changing the parameters of the two sets of features,compared with the single feature before fusion,the new feature after fusion can get a higher recognition rate.(2)Using deep learning to automatically extract micro-expression features,designing a shallow Two-Stream Spatial-Temporal Network.Using Three-dimensional Convolutional Neural Network and Convolutional Long and Short-Term Memory Network extract micro-expression sequences features at the same time,and adding the Dropout algorithm to the framework to prevent overfitting.Finally,the extracted two features are fused through the concatenation layer.Before the micro-expression sequence is input into the network,the corresponding pre-processing is done,including the database sample expansion,normalize the frame number,and the micro-expression image sequence is converted into the texture sequence.Experiments performing on SMIC and CASME? micro-expression data,the results show that the fused features can achieve better performance.(3)On the basis of the framework proposed in work(2),a micro-expression prediction system was built.The system functions include emotion prediction of the micro-expression sequence in the corresponding database;display the change of the predicted image sequence during the convolution process;display model the loss function curve during training.