Research On The Technology Of Facial Expression Recognition In Video Sequences

Facial expression recognition is to analyze and detect the special expression state from given expression images or video frames and then to ascertain the subject's specific inborn emotion, achieving smarter and more natural inter-action between human beings and computers. The study of facial expression recognition based on video frames has found important realistic values.In this work, we firstly discuss the background and then analyze the main expression recognition algorithms presented, emphasizing on Principle Components Analysis(PCA), wavelet transformation, Optical Flow Models(OFM) and Hidden Karkov Models(HMM). In a conclusion, the methods of facial expression recognition can be fallen into two classes: static recognition based on still facial expression images and dynamic recognition based on video frames. The static recognition methods are simple and quick, though they generally require images to reflect an exaggerated intensity of expression. The dynamic methods deal with every frame image in video frames so they have universality. But the dynamic models are complex and their computation is very expensive. So they can't meet the demand of real time generally.In order to meet the requirement of real-time ability and authenticity on expression recognition from video frames, we present in this work the expression key-frames detection algorithm from video frames based on images differentiating secondly. And then we propose the expression features extraction algorithm and recognition algorithm from still key-frames, which can achieve real-time facial expression recognition effectively. The methods are discussed as below:(1)Gabor wavelet transformation is performed on every frame in the video to extract the expression features of this frame. The scale normalization and grayscale equalization on the frontal facial images are helpful for extracting expression features more effectively. After transformation, a set of vectors formed by wavelet coefficients present expression features, which are called elastic graph for expression features.(2)Through tracking a series of elastic graphs, the key-frames , which contain expression features in exaggerated intensity, can be detected based on images differentiating.(3)The expression key-frames are viewed as typical images. Elastic templates matching algorithm and promoted K-nearest neighbors classifier are proposed to recognize these typical expression images.In this work, the static and dynamic recognition methods are combined so that the number of testing expression images are cut down greatly and the computation cost is reduced. Moreover, the expression extraction algorithm based on Gabor wavelet transformation can shield from the influence of illumination and subjects' variety. Therefore, the subject-independent facial expression recognition can be achieved.In the end of this work, we design a prototype system of facial expression recognition based on object-orient methods and prove the effectivity of above algorithms by some experiments.