Facial Expression And Micro-expression Recognition Based On Multi-feature Fusion

Facial expression contains human complex inner feeling which is an important way to communicate among persons. In recent years, it has become the focus of research in many fields such as computer vision, human-computer interaction, pattern recognition, and so on. Ordinary facial expressions are usually regarded as exaggerated ones which show big facial muscle movement and easy to be detected by naked eyes. However, micro-expression is a sort of subtle expression. It is discovered by Ekman first, due to its short duration, so it is named as micro-expression.The facial expression and micro-expression have low recognition rate. So this paper proposes a novel facial expression and micro-expression method based on Haar and LBP features. The main work of this paper is as follows:Firstly, in order to eliminate some interference conditions such as uneven illumination and different scales. This paper contains facial region segmentation, histogram equalization, contrast enhancement and scale normalization.Secondly, feature extraction. This is the most important step of expression and micro-expression recognition. This paper fuses the Gabor and partition HLBP for static expression and micro-expression, and HLBP algorithm is extended to HLBP-TOP which is applied in micro-expression recognition of image sequence. The former mainly relates to the direction and scale of the Gabor filter and the threshold values selection of Haar. This paper compares multiple sets to choose the optimal value of classification. The latter optimizes Haar threshold through statistical theory based on the former. The fusion method is better than any single algorithm and retains the useful information of images.Finally, classification and recognition: This is the last step. This paper chooses extreme learning machine and support vector machine to classify the expression images.There are two databases used in the experiment, JAFFE and CASME databases. In order to verify the validity of the algorithm, for CASME database, this paper establishes the sequence expression and static expression datasets respectively. The static expression database and JAFFE database are used in the facial expression recognition. The experimental results show that the algorithm has a better performance than the single Gabor and LBP algorithm. And for micro-expression experiments on the image sequences, the experimental results are superior to LBP-TOP and DTSA algorithm, and time efficiency is much higher than LBP-TOP algorithm.