Research On Algorithms Of Feature Extraction For 3D Facial Expression Recognition

The facial expression is one of the primary carriers for understanding and analyzing human emotions. The aim of facial expression recognition is to teach the computers to learn human emotions, so that they can recognize, understand and express emotions like human beings. With the increasing requirements of machine intelligence, this research topic has aroused widespread concern from domestic and foreign institutions.The classical methods for research on 2D facial expression recognition have obtained good recognition results, but there are still unresolved problems, such as the variation of illuminations and facial poses, which are caused by the intrinsic attributes of 2D images, so they cannot be well solved by 2D images.3D facial expression is the three-dimensional shape information of facial expessions, which will not be affected by the change of environments, so they can be used to avoid the problems of 2D facial expression recognition to obtain better recognition results. Based on the abortive analysis of 3D facial expressions, this paper focuses on the research of algorithms for feature extraction. The main contributions are listed as follows:1. The angle features are proposed, and their fusion with distance features is introduced to describe the facial expressions preciselyBased on the comparison of different facial expressions defined in FACS (Facial Action Coding System), the angle features are extracted to describe the evident variation of facial expressions. The classical distance features are employed to describe the displacement of local regions;the angle features and distancefeatures describe different attributes of facial expressions, respectively, so the relativity between them is limited. As a result, their fusion is further proposed to obtain more details of facial expressions, so that the facial expressions can be described more precisely. Based on the experimental results, the angle features are guaranteed to be applicable to facial expression recognition, and the recognition results can be improved by their fusion with distance features.2. PCA relative entropy is proposed to measure the discriminant power of featuresBased on the Bayes Theory, the discrimination power of features depends on their heterogeneous conditional probability distributions, and the relative entropy is always used to measure the difference between two sets of probability distributions. The symmetry of relative enrtropy is improved based on arithmetic mean algorithm to meet the demand of measurement, and the PCA algorithm is integrated to ensure the universality ofPCA relative entropy. The computation complexity for this criterion is low, and it can be efficiently used to choose the facial expression features with higher discriminative power. In the experiments, different sets of facial features are extracted and different classifiers are involved for facial expression recognition, the results show that the discriminant power of features are directly proportional to their amount of PCA relative entropy.Therefore, the PCA relative entropy is verifiedto be the benchmark for measuring the discriminant power of 3D facial expression features.3. The average neutral face is proposed to generate the delta faces of depth imageThe method of 3D rasterization is firstly proposed to transform the 3D mesh faces into grid structure, and then the average neutral face is generated, so that the part of neutral faces in original 3D faces is removed, and then the corresponding depth images are finally obtained. As this kind of depth images have removed the component of neutral faces, only the residual components of facial expressions are used to express the variation of facial expressions, the features extracted from them are of better class-separability and subject-independence. The experimental results show that the delta faces of depth images, which are obtained based on the average neutral face,not only keep the main variation of the facial expressions, but also keep the relative intensity well and the main variation of facial expressions in depth images still focuse on local regions. This kind of depth images can express the intrinsic variations of facial expressions well, so that the results of facial expression recognition are promoted. These results further validate the practicability of depth images for the research of 3D facial expression recognition.4. IreEnLBP is proposed as a novel feature to describe 3D facial expressionsAn "image-like grid structure" is firstly proposed, and 3D facial expressions are preprocessed based on this structure, so that the classical features can be used to describe the variation of 3D facial expressions. Then,"the irregular division schemes" and "the entropy weighted algorithm" are proposed, and they are applied to LBP, sothat IreEnLBP is proposed asa novel feature to describe 3D facial expressions precisely. IreEnLBP not only has the advantages of LBP, but also utilizes"the irregular division schemes", so that the faces are divided following the distribution of the main organs, which keeps the intensity of local organs. Therefore,different blocks are obtained corresponding to different facial expressions, and the class-separability of facial expression features is enhanced. Moreover, the feature of each block is weighted based on its entropy, so that the IreEnLBP features represent the influence of local regions for different facial expressions and theiruniqueness is strengthened. The experimentsshow that the recognition results of 3D facial expression can be sharply improved using IreEnLBP.