Facial Expression Recognition Based On Local Features Analysis

With the continuous development of computer science and technology, human-computer interaction technology is gradually becoming a hot research field of artificial intelligence, and the issue of emotional communication between human and computer has caused wide public concern. If computers and robots have the ability to understand and express emotions as human beings, to judge the initiative and help people complete tasks according to one's environment, emotions, hobbies, habits and other information, that will change the relationship between human and computer fundamentally, and make the computer to serve humanity better. Therefore, how to make an in-depth study on facial expression recognition and let computer to understand and express emotions has become the needs of era. This can not only promote the development of relevant subjects, but also provide theoretical support for the security surveillance, video communication, the medical field, games, entertainment and other fields.In recent years, although facial expression recognition technology has made some progress, it is still an exploring research overall. To a machine, there is a gap for it to imitate facial expression recognition of human conducts. Especially in automatic facial expression recognition system, there are still many theoretical and technological issues need to be resolved, such as facial image preprocessing, facial intrinsic feature extraction, recognition accuracy and real-time performance, as well as the identification ability for the upper complex expressions, et al.. In this article, facial feature extraction related methods are studied, and some new algorithms of face detection and preprocessing, eye location and facial expression recognition are proposed. Such as after analyzing face detection and facial expression image preprocessing method, it implemented face detection based on skin color detection and AdaBoost methods, designed the face image normalization strategy; It researched precise human eye locating methods, proposed a regional projection methods and an adaptive ratio local binary pattern method to achieve eye location; To the facial expression recognition problem, it proposed a method based on feature blocks and local binary pattern to build a feature model, realized the data fusion of various components by using D-S evidence theory, and made judgments to the fusion results.The specific research works and results include the following aspects:(1) The traditional methods of face detection and preprocessing are studied, and a new face detection algorithm is proposed based on skin color and AdaBoost. Firstly, the Haar-Like feature is used for training classifier. Then, skin color is adopted to detect candidate face regions roughly. Finally, the AdaBoost-based algorithm is used for accurate detection. Besides, the methods of grayscale normalization and geometric normalization are analyzed for facial expression images. It can obtain the facial expression region which is beneficial to expression analysis from original image, and prepare for subsequent expression feature extraction and recognition.(2) To the poor anti-jamming ability of traditional gray projection method, we propose a new method based on region projection. Firstly, taking into account the two-dimensional characteristics in the process of projection, we divide the eye image into several non-overlapping regions, and obtain the candidate region of pupil by horizontal projection and vertical projection, expand the region and get a pupil window. Then, we locate the pupil center exactly by border tracking with the use of the gray characteristics. Finally, we work out a criterion to evaluate the location accuracy of human eye location, and test the algorithm in Caltech faces databases and JAFFE database. Experiment shows that the method is effective and robust, has higher eye-location precision than traditional projection methods.(3) Combining the grayscale distribution of eye area, a new feature extraction algorithm called adaptive ratio local binary pattern (ARLBP) is proposed based on the analysis of traditional local binary pattern. Firstly, the principle of obtaining adaptive ratio is described in details, in which the pixel value ratio of center point and it neighbors is analyzed by computing the histogram. Then, the coarse positioning method and precise eye positioning method are combined for located the eye center. Finally, the system is tested based on two standard facial databases, experimental results show that the method is simple and effective, can able to extract the gradient information of human eye, eyebrows and other areas, and has a certain adaptive ability to light changing.(4) To the question of facial expression recognition, a novel method is presented based on feature blocks and LBP descriptor. The method introduced here utilizes the well known framework of linear space analysis. Firstly, the facial image is divided into many regions, taking into account the importance, the eyes block and mouth block are used for expression analysis. Then, the eye block and mouth block are divided into several overlapping sub-regions equally to extract LBP histograms. After that, the principal component analysis (PC A) method is used to learn the structure of the expression in the LBP feature space. Finally, the recognition experiment is conducted on the JAFFE facial expression database and TFEID database by using the nearest neighbor classifier. Experimental result shows the competitive performance.(5) To the questions of feature selection and multi-feature fusion in facial expression recognition, a model is proposed in this paper based on evidence theory and local texture characterization operator. Firstly, the facial image is divided into several areas with significant recognition features and the area local binary pattern texture features are extracted. Then, the LBP histograms in the local regions are connected into a single histogram graph and Chi-square distance is used as the similarity measure to establish the guidelines for evidence synthesis. Finally, D-S evidence theory is adopted to finish the feature vector data fusion of all components and the judgment of expression class is carried on. Experiment shows that the method is simple and effective, has high recognition rate and can improve the performance of facial expression recognition system to some extent.The main work in this thesis can be regarded as some useful exploration and attempt for enriching the existing facial expression recognition algorithms. Experimental results show that the proposed algorithms can basically meet the requirements of practical application to some extent. Based on this work, if further deep researches can be conducted to improve the algorithms'performance, it will continue to optimize the automatic facial expression recognition system.