Research On Expression Recognition Based On Multiple Features Ensemble And Rating Classification For Image Sequences

Facial expression recognition is to analyze and detect the special expression state from given expression images or video frames and then to ascertain the subject's specific inborn emotion, achieving smarter and more natural interaction between human beings and computers. The study of facial expression recognition has found its values in economy and society.In this thesis, we firstly discuss the background and then analyze the main exiting expression feature extraction algorithms and expression recognition algorithms. After analyzing the methods currently used by others, we present the expression recognition method based on multiple features extraction and classifiers ensemble, and the method of subtle expression recognition based on rating classification for image sequences. The main work is described as bellows:(1) Expression recognition method based on multiple features extraction and classifiers ensemble is presented. The feature expressive manner of emotion information is difference at distinct facial position. In this method, texture features for the eye area are extracted by using Gabor wavelet transformation, texture features for the nose area are extracted by using 2D-DCT, and shape variety features for the mouth area are extracted by using AAM. Following the Bagging principle, discrete HMM is adopted for the areas of eye with nose, mouth with nose and mouth with eye by exploiting repeatedly the features in these facial areas. Then the component classifiers are combined into a strong classifier in order to acquire more expression information and enhance the ability of classification. In the process of recognition, recognition result is acquired by combining the voting-based method with multiple features with the weight of each repeatedly used features obtained by contribution analysis algorithm. Experiments show that the method can improve the recognition rate.(2) The method of subtle expression recognition based on rating classification for image sequences is presented. In expression categories, some expressions are similar and difficult to be differentiated, whereas some others are different and easier to be differentiated. The expressions are classified roughly by subsuming confusable expressions in the same kind of expression firstly. Then the expression features that contribute much to this kind expression are selected for fine classification. In the process of each grade classification, discrete HMM is adopted for expression recognition in each expression area respectively. The classification results are fused by means of integrating the probability of each expression in each area with its weight obtained during the training phase. Experiments show that the method can recognize subtle expression effectively.(3) A prototype system of facial expression recognition based on image sequences is designed and implemented by using the object-oriented methods. The system consists of the subsystem of facial expression recognition based on multiple features extraction and classifiers ensemble and the subsystem of subtle expression recognition based on rating classification. It can be used to judge the expression category for image sequences.