Study On Some Key Issues In Facial Expression Recognition

Automatic facial expression recognition (AFER),which is developed to recognize one of our human special emotional representation–facial expression, has been attracted more and more attention. It is full of challenging and difficult because of the complicated and subtle properties of facial expression. For a whole AFER system, the difficulty and challenges are showing: (1) how fast and accurate to get a face image (2) how to alignment the facial features with high accuracy (3) how to extract and recognize facial expression features. In this thesis, all these problems in AFER are researched and the details are:1. After briefly introducing the basic emotion theory, this thesis provides a thorough survey of the AFER history and the state-of-the-art. Firstly, the state of affection computing research in the world is discussed. Then the basic emotion theory is introduced. After that, this thesis put more emphasis on a more recent overview of the AFER research and development. Started from the discussion of the general computational model of AFER methods, the AFER researches are partitioned to three historical stages according to its research method and target. For each stage, its technical characteristics are summarized and typical methods are detailed described. Then AFER methods are further categorized according to facial feature extraction, face representation, and classification separately. Following, the main public facial expression databases and main AFER systems are also surveyed as well with their technical sources. Finally, the challenges and technical trends in AFER fields are discussed.2. Face detection using skin-color detection and gradient template matching are studied. Face detection is very important in face image research and there are many classic methods for detection with gray images, but on the other hand, detection using skin-color is comparably novel. Base on the research result of face skin-color space by other researchers, a new gradient template is adopted, in which more face shape information is contained. The experiments results show that the combination of two methods has the effect of wiping off much non-skin-color area with skin-color model and detection face with high accuracy and speed using our gradient template.3. Search subspace and search process in classic ASM used for accurate facial feature alignment are improved. Accurate facial feature alignment is the prerequisite of an AFER system. Active Shape Model (ASM) is the main method for this problem. After analysis its work principal and its merits and demerits all-sided, three kinds of improvement are proposed: (1) Add the shape varying subspace to the original shape subspace to improve its shape reconstruction ability. The mixture shape subspace is called the optimal searching shape subspace in this paper. (2) Since it is independent between shape reconstruction and point searching in classic ASM, a reasonable, simple and convenience evaluation function is proposed to evaluate shape searching result every step and finally the whole searching process is not blinded any more and a more reasonable searching result would be acquired. (3) Based on the second evaluation function improvement, an optimal search method is proposed to make the searching results in both shape space and image texture space are restricted and used each other with feedback mechanism, so that the searching process is more reasonable and intelligence. All these improvements enhance the shape subspace and searching process of the classic ASM in some degree and give a more accurate facial feature alignment result.4. Face representation using independent components is studied and a new method of AdaICA facial expression recognition method is proposed. ICA, because its components are independent and it use high order statistic information, has much superior in making discrimination analysis. But it still has two shortcomings: one is that the acquired independent components are random and the other is the discrimination functionality of estimated independent components is not stable, some are very strongand some are very weak. To solve such problems, in this paper, a new method called AdaICA is proposed to get those independent features which have strong discrimination ability through running ICA several times and boosting the selected independent components. A series of experiments were tested on JAFFE and our selected database. The results indicate that the features extracted by AdaICA have more discriminating ability than other features and the recognition rate is improved greatly.5. Global facial expression feature extraction and analysis method using DWT and DCT is studied. In order to test global feature performance, wave packet decomposition is used to compress original images and discrete cosine transform is carried out to remove correlation and accumulate the energy from the low frequency data generated by discrete wavelet packet decomposition. Finally the data of diagonally are extracted to be as recognition features. The proposed method is very simple and convenience to realize.6. In the view of local feature extraction method, a new local binary patterns facial expression recognition method is proposed. Since facial expression is formed by series of move of muscles on the face, accurate local features extraction should be very competitive. But one the other hand, following points should be considered: at first, the extracted features are really local features which can reflect local space information, not just from the global method in local area. Second, they are should be very sensitive to small changes, which can ensure extract enough facial expression detailed information. Third, they can be combined casually in some degree because facial expression is the result of local transform combination. Based on such consideration, a strong texture description operator used in computer vision called local binary patterns is introduced. In this paper, the classic LBP method is enhanced from three aspects for facial expression recognition: image data, constructing feature way and the way of combining all extracted features. At first, to increase the original images data, wavelet packet is used to decompose images into four kinds of frequency images and the very four images are reconstructed separately with other frequency images be zero. Then LBP data on these four kinds of images are extracted in local and holistic way, which will make the features contain the global and local information. At last, we combine all data with adaptive weight mechanism according to the images content. Every kind of experiments is also proved that the proposed improvements in this paper have promoted the performance of facial expression recognition greatly.