Facial Expression Recognition Based On Fusion Of Multi-view Information

With the widespread use of computers, especially the popularity of variousterminals for mobile devices, and the rapid development of the technologies asartificial intelligence and computer vision, it gains more and more attention to realizea natural human behavior perception, such as the facial expression recognition.Facial expression recognition has a potential wide range of applications in the field ofhuman-computer interaction, basic science research and practical business. In fact,facial expression recognition is not only a hot topic, but also a challenging researchtopic with the recognition influence caused by the factors such as diversity andcomplexity of natural data. Most current methods mainly used standard sampleswhich are collected under the experimental environment, viz. frontal or similar frontalfacial expression images to decrease the complexity of samples. In order to increasethe ability of facial expression perception from the perspective of multi-view, thepaper makes deep research on the multi-view of facial expression recognition and thefeature extraction. The main research work and innovations are introduced as follows:1. This paper proposes an edge detection approach based on edge weight fusion.According to the characteristics of the edge point with sharp change, the paperproposes two kinds of edge weight matrices to describe the extent of a pixel being tothe edge, which are edge distribution weight matrix and difference weight matrixunder the analysis of edge spatial structure attribution. Then the fusion strategy isemployed in edge detection based on the two matrices. To evaluate the performance ofour proposed edge detector, comprehensive quantitative and qualitative experimentsare done. The results show that our proposed edge detector could effectively improvethe performance of edge detection, especially alleviating missing edge rate and falsealarming rate. The proposed method effectively retains the edge information anddecreases the interference of noise largely.2. This paper proposes a kind of facial expression feature derived from edge.Under the review of typical methods of facial expression feature extraction, the paperdiscusses the discriminant ability of edge as a geometric feature to represent facialexpression. The specific process is as follows. Firstly, get edge points by proposededge detector, then count the number of edge points along row and column, andobtained numbers are concatenated into our feature vector, which effectively reservesthe position relationship of edge pixels in the original image. To test the effectivenessof this edge-based feature, the paper uses it in the facial expression recognition.Several classifiers are adopted in the paper, which are K-NN, SMO and Bayes Net.The experimental results demonstrate that the proposed edge-based feature has goodexpression result and the feature is obtained efficiently under this simple and fast feature extraction method.3. Study the profile facial expression feature based on SIFT. In view to theprofile faces from the multi-view have the characteristics which are difficult to detect,high noise and instability, this paper uses sift feature to extract the profile facialinformation and build profile facial expression features, to facial expressionrecognition. This method overcomes the effect of profile face because of thedifference of perspective, and is an efficient complement of frontal face, and thenprovides the basis of facial expression recognition.4. This paper proposes a facial expression recognition system based onmulti-view information fusion. Consider the lack of multi-view facial expressionpublic image database, this paper finished building a multi-view facial expressiondatabase, and did the experiment to verify our system that facial expressionrecognition based on the fusion of multi-view information. This system is: featureextraction in multi-view image data, and then fuse in the feature layer, and finally useclassification to recognize them. The experiment shows that our system can get agood result, that is, the recognition rate of fusion result of the multi-view images hasbetter performance than that of the single image.