Study On The Video-based Facial Expression Recognition Technology

The study of intelligent human-machine interaction has gained more and moreattention along with the development of friendly requirements of the people to thecomputer man-machine interface. Facial expression recognition is an important way ofhuman-computer interaction. Through the analysis of facial expression we can know thefeelings of human. Thus we could make the intelligent human-machine interactionreality.There two types of facial expression recognition methods, the image-based methodand video-based method. Hidden Markov Model (HMM) combines spatial and temporalinformation, and has a good recognition effect. To simplify the process, current HMMmethods usually use fixed number of states, and take uniform distribution frame interval.While, the rhythm of expression movement has apparently individual difference, theexpression intervals got above don’t match the true expression changes well. Thoseimpact the performance of HMM seriously.Image-based facial expression recognition uses face texture features; it is a goodcomplement to the movement characteristics. But the traditional methods cannotguarantee the best expressions plumpness, which will affect the recognition results. Weconsider to analyze the variation tendency of expression of video to extract face imageframe corresponding to the best full expressions plumpness to improve recognitionresults.This paper designed an expression recognition method combining static anddynamic characteristics. The main work is as follows. Firstly, expression segmentation.The method determines the starting and ending frame of each expression automaticallyby analyzing the change curve of movement energy of facial features. Secondly,extraction of state number and state frame. The method determines the number of statesand the frame corresponding to the state by statistical analysis of movement energyvariation for different facial area. Thirdly, training and recognition on HMM. The HMMmodel uses geometric characteristics in training and recognition. Lastly, static facialexpression recognition based on full-expression facial images. We extract Gabor texturefeature of the facial image when the facial expression is full, and use the elasticmatching template to recognize expression. We get the final recognition results byweighting the static facial expression recognition results and HMM identification results.Algorithm also implemented the face detection and feature points’ extraction andtracking. Face detection uses Haar feature and AdaBoost algorithm. Feature points’extraction uses morphological method, projection analysis, corner detection, opticalflow and so on. Detection and tracking of feature points provide the data for the motionenergy analysis.Experiments used the USTC-NVIE face database. The recognition results areapparently improved compared with the HMM expression recognition which has singledynamic characteristic of expression.